Nov 03 20:38:33.174874 ip-10-0-141-248 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 20:38:33.576028 ip-10-0-141-248 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:33.576028 ip-10-0-141-248 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 20:38:33.576028 ip-10-0-141-248 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:33.576028 ip-10-0-141-248 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583499    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583598    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583602    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583605    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583608    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.583601 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583611    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583614    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583616    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583619    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583622    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583625    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583628    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583630    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583633    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583635    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583638    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583641    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583643    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583646    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583648    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583651    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583654    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583656    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583659    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.583827 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583661    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583664    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583667    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583670    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583673    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583675    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583677    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583680    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583683    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583686    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583688    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583691    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583693    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583695    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583698    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583701    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583704    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583708    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583713    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.584257 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583716    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583719    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583722    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583724    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583727    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583730    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583733    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583735    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583738    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583740    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583743    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583746    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583748    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583751    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583754    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583757    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583759    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583761    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583764    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583768    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.584715 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583772    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583775    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583778    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583780    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583783    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583786    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583788    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583791    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583793    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583796    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583798    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583801    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583803    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583806    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583808    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583811    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583813    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583816    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583818    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.585188 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583820    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583824    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583826    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583829    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583831    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583834    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583837    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583840    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583842    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583845    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583847    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583850    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583853    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583857    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583859    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583862    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583864    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583867    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583869    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583872    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.585664 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583874    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.586128 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583877    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.586128 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583880    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.586128 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.583882    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.586287 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586274    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586287    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586291    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586294    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586298    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586301    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586304    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586307    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586309    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586312    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586315    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.586317 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586319    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586324    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586327    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586330    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586332    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586335    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586337    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586340    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586343    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586345    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586348    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586351    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586353    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586355    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586358    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586360    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586363    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586365    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586368    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.586603 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586370    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586373    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586375    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586378    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586380    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586383    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586385    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586387    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586390    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586392    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586395    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586398    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586400    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586403    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586406    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586409    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586412    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586415    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586417    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586420    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.587052 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586422    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586425    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586428    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586430    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586433    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586435    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586438    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586440    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586442    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586445    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586447    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586450    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586452    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586454    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586457    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586461    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586465    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586468    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586470    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.587569 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586473    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586475    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586478    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586488    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586491    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586494    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586496    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586499    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586502    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586505    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586508    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586510    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586513    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586515    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586518    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586520    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586522    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586525    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586528    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586530    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.588024 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586533    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586535    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586538    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586540    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586543    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586545    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586548    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586550    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586566    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586569    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586572    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586574    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586577    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586579    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586582    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586586    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.586589    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587078    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587084    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587088    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.588515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587091    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587094    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587096    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587099    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587102    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587104    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587107    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587109    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587113    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587117    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587120    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587123    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587126    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587129    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587131    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587134    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587137    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587140    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587142    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.589006 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587145    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587147    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587150    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587152    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587155    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587157    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587160    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587162    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587165    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587168    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587170    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587173    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587175    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587178    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587181    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587184    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587186    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587188    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587191    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587193    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.589593 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587196    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587198    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587200    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587203    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587206    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587208    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587211    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587213    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587215    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587218    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587221    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587223    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587225    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587228    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587230    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587233    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587235    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587238    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587242    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.590103 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587246    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587249    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587252    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587255    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587257    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587260    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587262    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587265    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587268    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587271    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587273    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587276    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587278    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587281    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587283    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587285    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587287    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587290    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587292    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.590539 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587295    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587298    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587300    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587303    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587305    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587308    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587310    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587312    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587315    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587318    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587320    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587322    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587325    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587327    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587330    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587332    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587334    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587337    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587339    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587341    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.590997 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587344    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587347    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587350    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587352    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587355    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.587358    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588280    2578 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588296    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588303    2578 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588308    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588312    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588315    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588319    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588323    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588326    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588328    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588332    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588335    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588338    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588341    2578 flags.go:64] FLAG: --cgroup-root=""
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588343    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588346    2578 flags.go:64] FLAG: --client-ca-file=""
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588349    2578 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 20:38:33.591502 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588352    2578 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588355    2578 flags.go:64] FLAG: --cluster-domain=""
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588358    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588362    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588364    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588368    2578 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588371    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588374    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588377    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588380    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588383    2578 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588386    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588389    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588392    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588395    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588400    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588404    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588407    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588409    2578 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588412    2578 flags.go:64] FLAG: --enable-server="true"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588415    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588420    2578 flags.go:64] FLAG: --event-burst="100"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588422    2578 flags.go:64] FLAG: --event-qps="50"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588425    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 20:38:33.592051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588428    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588431    2578 flags.go:64] FLAG: --eviction-hard=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588435    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588438    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588441    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588444    2578 flags.go:64] FLAG: --eviction-soft=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588448    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588450    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588454    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588456    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588459    2578 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588462    2578 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588465    2578 flags.go:64] FLAG: --feature-gates=""
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588469    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588471    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588474    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588477    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588480    2578 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588483    2578 flags.go:64] FLAG: --help="false"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588486    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588490    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588493    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588495    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 20:38:33.592699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588499    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588503    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588506    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588509    2578 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588511    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588514    2578 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588517    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588521    2578 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588524    2578 flags.go:64] FLAG: --kube-reserved=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588527    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588530    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588533    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588535    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588538    2578 flags.go:64] FLAG: --lock-file=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588541    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588544    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588546    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588552    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588572    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588577    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588582    2578 flags.go:64] FLAG: --logging-format="text"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588585    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588588    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588591    2578 flags.go:64] FLAG: --manifest-url=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588594    2578 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 20:38:33.593276 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588599    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588602    2578 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588605    2578 flags.go:64] FLAG: --max-pods="110"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588608    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588611    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588614    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588617    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588620    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588623    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588626    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588633    2578 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588636    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588639    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588642    2578 flags.go:64] FLAG: --pod-cidr=""
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588644    2578 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588647    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588650    2578 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588653    2578 flags.go:64] FLAG: --port="10250"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588655    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588658    2578 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0922fba4f01df7461"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588661    2578 flags.go:64] FLAG: --qos-reserved=""
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588664    2578 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588667    2578 flags.go:64] FLAG: --register-node="true"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588670    2578 flags.go:64] FLAG: --register-with-taints=""
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588674    2578 flags.go:64] FLAG: --registry-burst="10"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588677    2578 flags.go:64] FLAG: --registry-qps="5"
Nov 03 20:38:33.593873 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588679    2578 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588682    2578 flags.go:64] FLAG: --reserved-memory=""
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588686    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588689    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588691    2578 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588694    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588697    2578 flags.go:64] FLAG: --runonce="false"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588700    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588703    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588706    2578 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588709    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588711    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588715    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588718    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588721    2578 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588724    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588726    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588729    2578 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588735    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588738    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588741    2578 flags.go:64] FLAG: --system-cgroups=""
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588744    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588749    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588751    2578 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588754    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 20:38:33.594482 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588758    2578 flags.go:64] FLAG: --tls-min-version=""
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588760    2578 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588763    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588766    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588769    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588772    2578 flags.go:64] FLAG: --v="2"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588776    2578 flags.go:64] FLAG: --version="false"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588780    2578 flags.go:64] FLAG: --vmodule=""
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588784    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.588787    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588948    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588951    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588954    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588957    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588961    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588965    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588969    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588972    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588975    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588977    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588984    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.595133 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588987    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588989    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588992    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588994    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.588998    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589000    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589003    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589006    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589008    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589011    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589014    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589016    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589019    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589022    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589024    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589027    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589029    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589032    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589034    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589037    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.595999 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589040    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589042    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589045    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589047    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589050    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589052    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589055    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589057    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589060    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589062    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589065    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589067    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589070    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589073    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589076    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589078    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589082    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589085    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589087    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.596846 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589090    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589100    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589103    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589107    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589109    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589112    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589115    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589117    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589120    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589124    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589127    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589130    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589133    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589137    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589139    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589142    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589145    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589147    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589150    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.597697 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589152    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589155    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589158    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589160    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589162    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589165    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589167    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589170    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589173    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589175    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589179    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589181    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589184    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589186    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589189    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589191    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589194    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589196    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589199    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589202    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.598170 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589204    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589207    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589211    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589214    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589216    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589219    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589222    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589224    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589226    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589229    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589232    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589234    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589237    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589239    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589241    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589244    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.598828 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589246    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.589275    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589311    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589315    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589320    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589324    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589327    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589332    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589334    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589337    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589339    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589341    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589344    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589346    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589349    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589351    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.599510 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589354    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589356    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589360    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589362    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589365    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589368    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589370    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589373    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589375    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589378    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589380    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589383    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589385    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589388    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589390    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589392    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589395    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589397    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589400    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.600247 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589402    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589405    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589409    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589412    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589416    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589420    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589422    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589425    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589428    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589431    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589433    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589436    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589438    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589441    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589443    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589447    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589449    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589451    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589454    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.600886 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589456    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589459    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589461    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589463    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589466    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589468    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589470    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589473    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589475    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589478    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589480    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589483    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589485    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589488    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589490    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589493    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589496    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589499    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589502    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589504    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.601399 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589507    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589510    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589512    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589514    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589517    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589519    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589522    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589524    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589529    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589531    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589534    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589539    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589542    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589545    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589548    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589551    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589566    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589569    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.602225 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589572    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589574    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589577    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589580    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589582    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589585    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589587    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589590    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589592    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589594    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589597    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589601    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589604    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589607    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589609    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.589612    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.597664    2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.597683    2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.598473    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.598484    2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 20:38:33.602956 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598594    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598603    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598608    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598613    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598618    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598624    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598628    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598633    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598637    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598642    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598646    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598651    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598655    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598659    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598664    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598668    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598672    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598677    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598681    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598685    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.603420 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598689    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598694    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598698    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598702    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598708    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598714    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598719    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598724    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598728    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598735    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598739    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598746    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598752    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598758    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598762    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598767    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598771    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598776    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598780    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.604042 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598784    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598788    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598793    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598797    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598801    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598805    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598809    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598813    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598818    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598822    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598825    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598829    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598833    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598837    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598841    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598844    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598848    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598852    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598856    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598861    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.604492 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598866    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598870    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598874    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598879    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598884    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598888    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598892    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598896    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598900    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598905    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598909    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598913    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598918    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598922    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598927    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598931    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598935    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598939    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598943    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598947    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.605035 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598951    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598955    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598959    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598963    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598967    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598972    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598976    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598980    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598984    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598988    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598992    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.598997    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599001    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599006    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599010    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599014    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599019    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599023    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599027    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.605594 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599031    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599035    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599040    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599044    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599048    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599052    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599056    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599060    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.599103    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599366    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599374    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599380    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599385    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599389    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599393    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599398    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.606040 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599402    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599406    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599410    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599414    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599418    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599423    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599427    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599431    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599435    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599441    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599445    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599451    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599458    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599462    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599468    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599472    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599477    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599481    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599485    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.606491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599490    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599494    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599498    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599502    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599506    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599510    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599514    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599517    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599521    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599525    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599528    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599533    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599537    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599541    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599545    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599550    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599572    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599577    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599581    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.607292 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599586    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599590    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599595    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599599    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599603    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599607    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599611    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599618    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599624    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599631    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599635    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599639    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599643    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599647    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599651    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599655    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599660    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599664    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599668    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599672    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.607787 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599676    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599680    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599685    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599689    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599694    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599698    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599703    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599707    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599712    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599716    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599720    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599724    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599729    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599732    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599736    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599740    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599744    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599749    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599753    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.608446 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599757    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599761    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599765    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599770    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599775    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599779    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599784    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599788    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599792    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599796    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599800    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599804    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599808    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599813    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599818    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599822    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599826    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599830    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599835    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599839    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.609121 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599843    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599848    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.599888    2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599938    2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599945    2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599950    2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599955    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599959    2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599963    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599967    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599971    2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599975    2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599979    2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599984    2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599988    2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599992    2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:33.609809 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.599997    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600002    2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600006    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600012    2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600017    2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600021    2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600026    2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600030    2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600034    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600039    2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600043    2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600048    2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600052    2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600058    2578 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600062    2578 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600066    2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600071    2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600075    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600079    2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:33.610347 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600084    2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600088    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600092    2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600096    2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600101    2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600105    2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600109    2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600113    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600117    2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600121    2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600126    2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600130    2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600134    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600140    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600146    2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600150    2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600154    2578 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600158    2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600164    2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:33.611069 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600170    2578 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600176    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600180    2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600185    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600189    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600193    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600198    2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600205    2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600209    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600215    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600219    2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600223    2578 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600228    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600232    2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600236    2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600241    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600245    2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600249    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600254    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600258    2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:33.611505 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600262    2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600266    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600270    2578 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600274    2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600278    2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600282    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600289    2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600293    2578 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600298    2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600302    2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600306    2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600310    2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600314    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600318    2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600323    2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600327    2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600331    2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600336    2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600340    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600345    2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:33.611990 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600349    2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600353    2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600358    2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600362    2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600367    2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600372    2578 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600376    2578 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600381    2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600385    2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600389    2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600393    2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600397    2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600401    2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600405    2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:33.600409    2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.600482    2578 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.603595    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.604612    2578 server.go:1014] "Starting client certificate rotation"
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.604722    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:38:33.612501 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.605416    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:38:33.612964 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.606957    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:38:33.612964 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.612411    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:38:33.624600 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.624584    2578 log.go:25] "Validated CRI v1 runtime API"
Nov 03 20:38:33.628309 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.628287    2578 log.go:25] "Validated CRI v1 image API"
Nov 03 20:38:33.629055 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.629030    2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 20:38:33.637896 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.637875    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7a5ca57b-1524-4e95-bfa5-03384ba88f98:/dev/nvme0n1p3 c5baa399-1031-4f77-b14d-ca276d36e0fe:/dev/nvme0n1p4]
Nov 03 20:38:33.637984 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.637895    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 20:38:33.642061 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.641952    2578 manager.go:217] Machine: {Timestamp:2026-11-03 20:38:33.641450006 +0000 UTC m=+0.362460239 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089183 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec291b516e41aa05dd420e29ba0208f0 SystemUUID:ec291b51-6e41-aa05-dd42-0e29ba0208f0 BootID:c08b650d-6d52-4690-b179-77a102fab2a0 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:15:05:d7:82:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:15:05:d7:82:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:a7:a4:04:58:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 20:38:33.642795 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.642785    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 20:38:33.642910 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.642897    2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 20:38:33.646468 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.646443    2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 20:38:33.646605 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.646470    2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-248.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 20:38:33.646649 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.646607    2578 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 20:38:33.646649 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.646614    2578 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 20:38:33.646649 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.646635    2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:38:33.647409 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.647399    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:38:33.648712 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.648702    2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 20:38:33.653202 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.653190    2578 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 20:38:33.653317 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.653308    2578 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 20:38:33.654091 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654082    2578 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 20:38:33.654121 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654096    2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 20:38:33.654121 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654107    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 20:38:33.654121 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654115    2578 kubelet.go:405] "Adding apiserver pod source"
Nov 03 20:38:33.654250 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654124    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 20:38:33.654484 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.654473    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:38:33.656919 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.656901    2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 20:38:33.664631 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.664612    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:38:33.665450 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665438    2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 20:38:33.665484 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665468    2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 20:38:33.665661 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665651    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 20:38:33.665697 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665667    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 20:38:33.665697 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665673    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 20:38:33.665697 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665678    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 20:38:33.665697 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665684    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 20:38:33.665697 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.665690    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 20:38:33.666246 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666238    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 20:38:33.666274 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666247    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 20:38:33.666274 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666255    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 20:38:33.666274 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666261    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 20:38:33.666274 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666268    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 20:38:33.666391 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.666276    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 20:38:33.667961 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.667952    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 20:38:33.667994 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.667962    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 20:38:33.671780 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.671766    2578 server.go:1263] "Started kubelet"
Nov 03 20:38:33.671884 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.671859    2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 20:38:33.671952 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.671914    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 20:38:33.671996 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.671969    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 20:38:33.673171 ip-10-0-141-248 systemd[1]: Started Kubernetes Kubelet.
Nov 03 20:38:33.673619 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.673477    2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 20:38:33.674406 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.674391    2578 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 20:38:33.677945 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.677923    2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 20:38:33.680750 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.680735    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 20:38:33.681325 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.681294    2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 20:38:33.681420 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.681387    2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-248.us-west-1.compute.internal\" not found"
Nov 03 20:38:33.682091 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682073    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 20:38:33.682091 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682075    2578 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 20:38:33.682235 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682101    2578 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 20:38:33.682370 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682353    2578 factory.go:55] Registering systemd factory
Nov 03 20:38:33.682404 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682373    2578 factory.go:223] Registration of the systemd container factory successfully
Nov 03 20:38:33.682444 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682404    2578 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 20:38:33.682444 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682430    2578 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 20:38:33.682615 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682600    2578 factory.go:153] Registering CRI-O factory
Nov 03 20:38:33.682654 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682620    2578 factory.go:223] Registration of the crio container factory successfully
Nov 03 20:38:33.682690 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682673    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 20:38:33.682726 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682692    2578 factory.go:103] Registering Raw factory
Nov 03 20:38:33.682726 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.682721    2578 manager.go:1199] Started watching for new ooms in manager
Nov 03 20:38:33.687316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.683780    2578 manager.go:319] Starting recovery of all containers
Nov 03 20:38:33.696151 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.696133    2578 manager.go:324] Recovery completed
Nov 03 20:38:33.700245 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.700230    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.701884 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.701868    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.701948 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.701901    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.701948 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.701913    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.702465 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.702452    2578 cpu_manager.go:226] "Starting" policy="none"
Nov 03 20:38:33.702465 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.702462    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.702541 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.702465    2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 20:38:33.702541 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.702482    2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 20:38:33.703106 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.703089    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.703154 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.703121    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.703154 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.703132    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.704674 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.704663    2578 policy_none.go:50] "Start"
Nov 03 20:38:33.704714 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.704680    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 20:38:33.704714 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.704690    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 20:38:33.706615 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.706605    2578 policy_none.go:44] "Start"
Nov 03 20:38:33.740217 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.740080    2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 20:38:33.744186 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744172    2578 manager.go:341] "Starting Device Plugin manager"
Nov 03 20:38:33.744260 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.744214    2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 20:38:33.744260 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744226    2578 server.go:87] "Starting device plugin registration server"
Nov 03 20:38:33.744476 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744463    2578 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 20:38:33.744532 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744477    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 20:38:33.744600 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744539    2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 20:38:33.744710 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744661    2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 20:38:33.744710 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.744671    2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 20:38:33.745125 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.745103    2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 20:38:33.745207 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.745157    2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-248.us-west-1.compute.internal\" not found"
Nov 03 20:38:33.819681 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.819652    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 20:38:33.821733 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.820900    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 20:38:33.821733 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.820917    2578 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 20:38:33.821733 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.820934    2578 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 20:38:33.821733 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.820966    2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 20:38:33.828408 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.828341    2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-248.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 20:38:33.828408 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.828357    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-248.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 20:38:33.828931 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.828198    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-248.us-west-1.compute.internal.18e4a37e1e697fd1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-248.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-248.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 20:38:33.671737297 +0000 UTC m=+0.392747531,LastTimestamp:2026-11-03 20:38:33.671737297 +0000 UTC m=+0.392747531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-248.us-west-1.compute.internal,}"
Nov 03 20:38:33.845484 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.845468    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.846180 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.846163    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.846262 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.846193    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.846262 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.846205    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.846262 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.846227    2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.899371 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.899344    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 20:38:33.899456 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.899387    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2284k"
Nov 03 20:38:33.921081 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.921060    2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"]
Nov 03 20:38:33.921140 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.921117    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.921921 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.921906    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.921968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.921940    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.921968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.921953    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.924196 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924184    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.924341 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924326    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.924375 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924357    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.924966 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924951    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.925046 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924969    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.925046 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924989    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.925046 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924991    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.925046 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.924998    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.925046 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.925007    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.927222 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.927205    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.927306 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.927242    2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:33.927945 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.927928    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:33.928032 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.927964    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:33.928032 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.927976    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:33.961013 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.960996    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-248.us-west-1.compute.internal\" not found" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.963268 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:33.963250    2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-248.us-west-1.compute.internal\" not found" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.983605 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.983583    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.983679 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.983607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.983679 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.983625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:33.986945 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.986927    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:33.988145 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:33.988130    2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:34.084025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.083963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.084025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.083992    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.084025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.084013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.084196 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.084067    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.084196 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.084073    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.084196 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.084113    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.087222 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.087204    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:34.088021 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.088006    2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:34.088396 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.088382    2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.181473 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.181443    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.263861 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.263840    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.266005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.265988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.324738 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.324720    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:38:34.324838 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.324810    2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:38:34.380348 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.380301    2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:38:34.604386 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.604363    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 20:38:34.604921 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:34.604465    2578 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.141.248:47156->3.147.231.34:6443: use of closed network connection"
Nov 03 20:38:34.604921 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.604532    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 20:38:34.604921 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.604599    2578 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 20:38:34.604921 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:34.604518    2578 event.go:368] "Unable to write event (may retry after sleeping)" err="unexpected error when reading response body. Please retry. Original error: read tcp 10.0.141.248:47156->3.147.231.34:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-141-248.us-west-1.compute.internal.18e4a37e2035d5ab  default   4720 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-248.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-248.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-248.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 20:38:33 +0000 UTC,LastTimestamp:2026-11-03 20:38:33.84619823 +0000 UTC m=+0.567208464,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-248.us-west-1.compute.internal,}"
Nov 03 20:38:34.654466 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.654417    2578 apiserver.go:52] "Watching apiserver"
Nov 03 20:38:34.681851 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.681822    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 20:38:34.903347 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.903306    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 20:33:34 +0000 UTC" deadline="2028-06-14 04:30:31.999632693 +0000 UTC"
Nov 03 20:38:34.903347 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.903340    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14119h51m57.096295405s"
Nov 03 20:38:34.911515 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.911462    2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 20:38:34.911785 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.911768    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-gft95","kube-system/global-pull-secret-syncer-xjbt9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj","openshift-dns/node-resolver-66w89","openshift-multus/multus-b8zc5","openshift-multus/network-metrics-daemon-ffxrp","openshift-network-operator/iptables-alerter-c8m6k","kube-system/konnectivity-agent-cks2d","openshift-cluster-node-tuning-operator/tuned-62lhn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-sl28h","kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal","openshift-image-registry/node-ca-4cjfz","openshift-multus/multus-additional-cni-plugins-vc9ll"]
Nov 03 20:38:34.914120 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.914105    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:34.914189 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:34.914173    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:34.916096 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.916083    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:34.918123 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.918099    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:34.920172 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.920156    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:34.922435 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.922415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.924861 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.924839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:34.924943 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:34.924922    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:34.927083 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.927068    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.929269 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.929251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:34.931450 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.931435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.933823 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.933639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.936008 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.935992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:34.936120 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:34.936101    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:34.938300 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.938284    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.968140 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.968106    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 20:38:34.968265 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.968143    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 20:38:34.968265 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.968148    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 20:38:34.968265 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.968165    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-czwwq\""
Nov 03 20:38:34.969655 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.969639    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:38:34.969859 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.969847    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nf42\""
Nov 03 20:38:34.969961 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.969947    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 20:38:34.970025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.969988    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 20:38:34.971647 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.971631    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 20:38:34.971736 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.971659    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 20:38:34.971785 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.971734    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xfms\""
Nov 03 20:38:34.974067 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.974055    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 20:38:34.974243 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.974212    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 20:38:34.974243 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.974239    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-kqp9s\""
Nov 03 20:38:34.974389 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.974293    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 20:38:34.974427 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.974411    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 20:38:34.978725 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.978707    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 20:38:34.978839 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.978825    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 20:38:34.979065 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.979051    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 20:38:34.983019 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.983005    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 20:38:34.988460 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.988539 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.988539 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:34.988539 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.988740 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.988740 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988648    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.988740 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.988740 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.988856 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvc7\" (UniqueName: \"kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:34.988856 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988799    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.988856 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988826    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4j8h\" (UniqueName: \"kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.988935 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:34.988935 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.988935 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:34.988935 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:34.988935 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988929    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.988977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989009    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989043    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.989078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989190    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989291 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcwcr\" (UniqueName: \"kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989357    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989392    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989474    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989494 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989650    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjdn\" (UniqueName: \"kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989807    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwqr7\" (UniqueName: \"kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.989926 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989921    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.989994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990134    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fns9\" (UniqueName: \"kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.990552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979r2\" (UniqueName: \"kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kx9d\" (UniqueName: \"kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990919    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990946    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.990991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991014    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991085    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r96ml\" (UniqueName: \"kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:34.991160 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:34.991643 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:34.991132    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.012685 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.012665    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 20:38:35.012793 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.012727    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 20:38:35.013462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.012758    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nnzhp\""
Nov 03 20:38:35.013462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.013154    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 20:38:35.013462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.013362    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 20:38:35.013462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.013383    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 20:38:35.013882 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.013864    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-29dtl\""
Nov 03 20:38:35.015608 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.015587    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 20:38:35.015708 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.015596    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:38:35.015708 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.015625    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 20:38:35.017607 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017592    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6k78\""
Nov 03 20:38:35.017704 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017634    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxrmv\""
Nov 03 20:38:35.017704 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017687    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 20:38:35.017704 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017725    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 20:38:35.017855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017740    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 20:38:35.017855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017737    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 20:38:35.017855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017761    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm4wl\""
Nov 03 20:38:35.017855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017787    2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 20:38:35.017855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.017837    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vgzvk"
Nov 03 20:38:35.092114 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.092212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:35.092212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.092212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.092212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092209    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092227    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092282    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092322    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092323    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092330    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96ml\" (UniqueName: \"kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092334    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.092462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092464    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092485    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092527    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092528    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092594    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092607    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092646    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092647    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvc7\" (UniqueName: \"kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4j8h\" (UniqueName: \"kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092743    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092742    2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092775    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:35.093215 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092808    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092876    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.092886    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.092973    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:35.592923903 +0000 UTC m=+2.313934144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.092970    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093025    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093009    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093103    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093140    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093162    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.093968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093182    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093254    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093299    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093413    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093462    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwcr\" (UniqueName: \"kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093640    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093649    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093678    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093776    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093828    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093869    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093892    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093920    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093931    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093951    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.095874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093974    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjdn\" (UniqueName: \"kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.093855    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094082    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094187    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094226    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094252    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094166    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqr7\" (UniqueName: \"kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094386    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094438    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.096604 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094450    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094478    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094613    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094669    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094733    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fns9\" (UniqueName: \"kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094775    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094798    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.097098 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094830    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094836    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094862    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094872    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094911    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979r2\" (UniqueName: \"kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094944    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094946    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.094966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095028    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.095036    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.095078    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:35.59506352 +0000 UTC m=+2.316073762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095076    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095126    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095189    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095274    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.097538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095312    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095347    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095420    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kx9d\" (UniqueName: \"kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095468    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095468    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.095849    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.096002    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.096052    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.096677    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.096682    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.097172    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.098013 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.097286    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:35.158605 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.158581    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:35.158719 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.158611    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:35.158719 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.158623    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:35.158719 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.158683    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:35.658665834 +0000 UTC m=+2.379676070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:35.159079 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.159016    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvc7\" (UniqueName: \"kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.160711 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.160690    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96ml\" (UniqueName: \"kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.160918 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.160871    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwcr\" (UniqueName: \"kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.161348 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.161327    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4j8h\" (UniqueName: \"kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.164645 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.164596    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fns9\" (UniqueName: \"kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.165288 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.164996    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-979r2\" (UniqueName: \"kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:35.165288 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.165229    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kx9d\" (UniqueName: \"kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.165288 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.165272    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjdn\" (UniqueName: \"kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.165461 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.165374    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqr7\" (UniqueName: \"kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.249877 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.249843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:38:35.256132 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.256109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:38:35.280783 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.280758    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-66w89"
Nov 03 20:38:35.286252 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.286233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b8zc5"
Nov 03 20:38:35.321312 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.321286    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:35.325022 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.325000    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:35.338916 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.338896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:38:35.340278 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.340255    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff27c4bf_c1d8_47f8_9737_24510e201615.slice/crio-e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64 WatchSource:0}: Error finding container e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64: Status 404 returned error can't find the container with id e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64
Nov 03 20:38:35.340652 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.340635    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3e66eb7ec8c0db50f85619df6599ce.slice/crio-69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0 WatchSource:0}: Error finding container 69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0: Status 404 returned error can't find the container with id 69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0
Nov 03 20:38:35.341491 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.341475    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda713afcbd6eef9b8d78d253711a83e69.slice/crio-943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd WatchSource:0}: Error finding container 943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd: Status 404 returned error can't find the container with id 943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd
Nov 03 20:38:35.342183 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.342021    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5084c67b_30c3_4ee8_a72c_a2f76b1c8c64.slice/crio-e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599 WatchSource:0}: Error finding container e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599: Status 404 returned error can't find the container with id e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599
Nov 03 20:38:35.347338 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.347323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:38:35.347547 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.347533    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:38:35.349442 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.349422    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb233d06_f91f_4db9_be7b_bb9966b67a75.slice/crio-2e245a61f3c2ac5ab759e2984215d715280af815827f81c11f543f6e366f7102 WatchSource:0}: Error finding container 2e245a61f3c2ac5ab759e2984215d715280af815827f81c11f543f6e366f7102: Status 404 returned error can't find the container with id 2e245a61f3c2ac5ab759e2984215d715280af815827f81c11f543f6e366f7102
Nov 03 20:38:35.357013 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.356995    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51834fed_df6e_444b_9c2d_4501e262df12.slice/crio-badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727 WatchSource:0}: Error finding container badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727: Status 404 returned error can't find the container with id badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727
Nov 03 20:38:35.375892 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.375871    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:38:35.384627 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:35.384606    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6d5af76_ea42_41d7_9920_db291e351c53.slice/crio-9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89 WatchSource:0}: Error finding container 9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89: Status 404 returned error can't find the container with id 9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89
Nov 03 20:38:35.597784 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.597743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:35.597955 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.597799    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:35.597955 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.597891    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:35.597955 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.597898    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:35.597955 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.597949    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:36.597936946 +0000 UTC m=+3.318947169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:35.598119 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.597962    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:36.597955603 +0000 UTC m=+3.318965825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:35.677299 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.677269    2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:35.698666 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.698637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:35.698785 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.698768    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:35.698871 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.698804    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:35.698871 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.698819    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:35.698956 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:35.698883    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:36.698864966 +0000 UTC m=+3.419875207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:35.825268 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.825185    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerStarted","Data":"9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89"}
Nov 03 20:38:35.826398 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.826374    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj" event={"ID":"51834fed-df6e-444b-9c2d-4501e262df12","Type":"ContainerStarted","Data":"badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727"}
Nov 03 20:38:35.827489 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.827462    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62lhn" event={"ID":"bb233d06-f91f-4db9-be7b-bb9966b67a75","Type":"ContainerStarted","Data":"2e245a61f3c2ac5ab759e2984215d715280af815827f81c11f543f6e366f7102"}
Nov 03 20:38:35.828577 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.828548    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cks2d" event={"ID":"ff27c4bf-c1d8-47f8-9737-24510e201615","Type":"ContainerStarted","Data":"e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64"}
Nov 03 20:38:35.829692 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.829670    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599"}
Nov 03 20:38:35.830752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.830734    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerStarted","Data":"943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd"}
Nov 03 20:38:35.831720 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:35.831701    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"fd3e66eb7ec8c0db50f85619df6599ce","Type":"ContainerStarted","Data":"69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0"}
Nov 03 20:38:36.020292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.020239    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:35 +0000 UTC" deadline="2028-04-12 15:49:08.650000638 +0000 UTC"
Nov 03 20:38:36.020292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.020282    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12619h10m32.629721307s"
Nov 03 20:38:36.149291 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:36.149266    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fef4f1_d40d_4a12_9e0f_371838f9ecfa.slice/crio-fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d WatchSource:0}: Error finding container fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d: Status 404 returned error can't find the container with id fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d
Nov 03 20:38:36.150333 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:36.150312    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f8781a_81f9_43f9_bd78_c5bcf5191fc0.slice/crio-7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58 WatchSource:0}: Error finding container 7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58: Status 404 returned error can't find the container with id 7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58
Nov 03 20:38:36.184399 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.184375    2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:36.229426 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:38:36.229401    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84298777_7d2a_4d90_80aa_0cd0c65e04f3.slice/crio-1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659 WatchSource:0}: Error finding container 1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659: Status 404 returned error can't find the container with id 1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.606076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.606133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.606275    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.606335    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:38.606313962 +0000 UTC m=+5.327324187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.606405    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:36.606471 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.606439    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:38.606427501 +0000 UTC m=+5.327437729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:36.708263 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.707396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:36.708263 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.708121    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:36.708263 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.708143    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:36.708263 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.708156    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:36.708263 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.708214    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:38.708194711 +0000 UTC m=+5.429204960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:36.822225 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.822191    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:36.822420 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.822325    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:36.822420 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.822413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:36.822546 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.822499    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:36.822642 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.822558    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:36.822714 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:36.822653    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:36.846394 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.846364    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66w89" event={"ID":"84298777-7d2a-4d90-80aa-0cd0c65e04f3","Type":"ContainerStarted","Data":"1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659"}
Nov 03 20:38:36.854963 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.854934    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4cjfz" event={"ID":"87fef4f1-d40d-4a12-9e0f-371838f9ecfa","Type":"ContainerStarted","Data":"fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d"}
Nov 03 20:38:36.856933 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.856860    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c8m6k" event={"ID":"66a0afd2-dcb3-48ab-905d-4e4abab2fe97","Type":"ContainerStarted","Data":"1635c73de0734221b3fdd22ba9a30731b33a8d1967bce0f0ee7675c47a951cc9"}
Nov 03 20:38:36.858827 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:36.858794    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b8zc5" event={"ID":"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0","Type":"ContainerStarted","Data":"7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58"}
Nov 03 20:38:37.021115 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:37.021070    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:35 +0000 UTC" deadline="2028-06-15 15:53:11.603121194 +0000 UTC"
Nov 03 20:38:37.021115 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:37.021112    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14155h14m34.582014252s"
Nov 03 20:38:38.623359 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.623222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:38.623359 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.623290    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:38.623905 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.623435    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:38.623905 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.623502    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:42.623483065 +0000 UTC m=+9.344493351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:38.623905 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.623582    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:38.623905 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.623641    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:42.62362699 +0000 UTC m=+9.344637231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:38.724916 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.724615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:38.724916 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.724782    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:38.724916 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.724801    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:38.724916 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.724815    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:38.724916 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.724872    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:42.724852141 +0000 UTC m=+9.445862380 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.821526    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.821684    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.821782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.821881    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:38.821938    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:38.822099 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:38.822010    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:39.899551 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:39.899307    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="fcf6bcbbb4070df8d6f605487becc08e0efc922a37d26aa8ef820203f212ee56" exitCode=0
Nov 03 20:38:39.900001 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:39.899627    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"fcf6bcbbb4070df8d6f605487becc08e0efc922a37d26aa8ef820203f212ee56"}
Nov 03 20:38:40.821950 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.821871    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:40.822097 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:40.822028    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:40.822164 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.822102    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:40.822207 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:40.822174    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:40.822264 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.822233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:40.822361 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:40.822311    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:40.928709 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.928668    2578 generic.go:356] "Generic (PLEG): container finished" podID="a713afcbd6eef9b8d78d253711a83e69" containerID="d82c6d0e520ceea9178599028b00a575ec79ed17577c6a78569d4dda72733bca" exitCode=0
Nov 03 20:38:40.929225 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.928759    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerDied","Data":"d82c6d0e520ceea9178599028b00a575ec79ed17577c6a78569d4dda72733bca"}
Nov 03 20:38:40.934302 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.934262    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"fd3e66eb7ec8c0db50f85619df6599ce","Type":"ContainerStarted","Data":"ed8ad571d6aa8d42b02367f727246458dfae63f28a53762af719d61e3360951a"}
Nov 03 20:38:40.953224 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:40.953194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cks2d" event={"ID":"ff27c4bf-c1d8-47f8-9737-24510e201615","Type":"ContainerStarted","Data":"98a6691f042e8be2ef8489c71b85b745471751750f25f6fc2f59e234027d8eac"}
Nov 03 20:38:41.301081 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.301016    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" podStartSLOduration=6.300995945 podStartE2EDuration="6.300995945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:41.176727403 +0000 UTC m=+7.897737652" watchObservedRunningTime="2026-11-03 20:38:41.300995945 +0000 UTC m=+8.022006195"
Nov 03 20:38:41.838887 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.838458    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:41.839086 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.839028    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:41.960002 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.959968    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerStarted","Data":"0a380862285ebe8b8fb2e55ca3b9d20324602747d28b1b56de270e4bb1a91604"}
Nov 03 20:38:41.960372 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.960031    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:41.960983 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.960957    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:38:41.964226 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:41.964174    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cks2d" podStartSLOduration=2.003393289 podStartE2EDuration="6.964157337s" totalImagesPullingTime="4.960764048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:41.302169881 +0000 UTC m=+8.023180131" watchObservedRunningTime="2026-11-03 20:38:41.964157337 +0000 UTC m=+8.685167587"
Nov 03 20:38:42.660282 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.660246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:42.660415 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.660310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:42.660476 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.660444    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:42.660527 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.660506    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:50.660492645 +0000 UTC m=+17.381502868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:42.660730 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.660712    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:42.660811 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.660779    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:50.660752013 +0000 UTC m=+17.381762238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:42.760991 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.760954    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:42.761140 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.761120    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:42.761189 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.761148    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:42.761189 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.761161    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:42.761263 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.761216    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:50.76119919 +0000 UTC m=+17.482209416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:42.821771 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.821693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:42.821934 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.821836    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:42.821934 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.821902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:42.822044 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.821980    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:42.822044 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.822032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:42.822165 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:42.822128    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:42.966364 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.966320    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj" event={"ID":"51834fed-df6e-444b-9c2d-4501e262df12","Type":"ContainerStarted","Data":"da5d6a63ab6db316a720f9330d2b7168e16c83796c5dfd0a15f85410db77a133"}
Nov 03 20:38:42.983963 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:42.983923    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4cjfz" event={"ID":"87fef4f1-d40d-4a12-9e0f-371838f9ecfa","Type":"ContainerStarted","Data":"302fdc4f60926b68e5e639ccba887bb7160ce1c28d690c15ca8005f4c2471b12"}
Nov 03 20:38:43.108056 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:43.107254    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" podStartSLOduration=8.107239296 podStartE2EDuration="8.107239296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:42.214543411 +0000 UTC m=+8.935553659" watchObservedRunningTime="2026-11-03 20:38:43.107239296 +0000 UTC m=+9.828249543"
Nov 03 20:38:43.108056 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:43.107420    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4cjfz" podStartSLOduration=2.7182705609999998 podStartE2EDuration="9.107414475s" totalImagesPullingTime="6.389143914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:43.106678237 +0000 UTC m=+9.827688484" watchObservedRunningTime="2026-11-03 20:38:43.107414475 +0000 UTC m=+9.828424743"
Nov 03 20:38:44.481499 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.481481    2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 20:38:44.758592 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.757597    2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T20:38:44.481500266Z","UUID":"112e9fd9-3689-4e26-a74b-45f51d2e8e31","Handler":null,"Name":"","Endpoint":""}
Nov 03 20:38:44.760199 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.759734    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 20:38:44.760199 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.759769    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.821754    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:44.821937    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.822315    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:44.822429    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.822502    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:44.822773 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:44.822586    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:44.991765 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:44.991667    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj" event={"ID":"51834fed-df6e-444b-9c2d-4501e262df12","Type":"ContainerStarted","Data":"914573ebdbf1e354e3dbe7667fdaa872305832cbafb4a9e05c79b73d1d57b270"}
Nov 03 20:38:45.997245 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:45.997006    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-66w89" event={"ID":"84298777-7d2a-4d90-80aa-0cd0c65e04f3","Type":"ContainerStarted","Data":"40b1b44ec5632881378435abc6c51596c25b643cd38c9da5da559175089b2f25"}
Nov 03 20:38:46.000498 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:45.999774    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="f682c0c5c8af87e7c68e54389bd1dee0f3a2f5889b6da9b008198fd08a24b31c" exitCode=0
Nov 03 20:38:46.000498 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:45.999874    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"f682c0c5c8af87e7c68e54389bd1dee0f3a2f5889b6da9b008198fd08a24b31c"}
Nov 03 20:38:46.005082 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:46.005058    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b8zc5" event={"ID":"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0","Type":"ContainerStarted","Data":"6124cf540ea3c22799429b5958b3252d677cd9037d6e59829a04c2552b26585e"}
Nov 03 20:38:46.305532 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:46.305469    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-66w89" podStartSLOduration=2.139285296 podStartE2EDuration="11.305448519s" totalImagesPullingTime="9.166163223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:46.130330481 +0000 UTC m=+12.851340728" watchObservedRunningTime="2026-11-03 20:38:46.305448519 +0000 UTC m=+13.026458767"
Nov 03 20:38:46.822118 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:46.822051    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:46.822284 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:46.822125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:46.822284 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:46.822250    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:46.822284 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:46.822283    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:46.822456 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:46.822349    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:46.822456 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:46.822426    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:47.010657 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:47.010605    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c8m6k" event={"ID":"66a0afd2-dcb3-48ab-905d-4e4abab2fe97","Type":"ContainerStarted","Data":"641bd7e65e34a74aa2a3aaa387d9680e7a229cd8e30e6dec647d26c3cac838f8"}
Nov 03 20:38:47.014399 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:47.014364    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj" event={"ID":"51834fed-df6e-444b-9c2d-4501e262df12","Type":"ContainerStarted","Data":"df8fac7906214242fe89178b281b49eb759351a6940cda0c5de91405e12a2efc"}
Nov 03 20:38:47.136878 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:47.136762    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-b8zc5" podStartSLOduration=3.274979779 podStartE2EDuration="12.136747105s" totalImagesPullingTime="8.861767326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:46.43572061 +0000 UTC m=+13.156730857" watchObservedRunningTime="2026-11-03 20:38:47.136747105 +0000 UTC m=+13.857757351"
Nov 03 20:38:47.137055 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:47.136995    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c8m6k" podStartSLOduration=3.073634945 podStartE2EDuration="12.136988166s" totalImagesPullingTime="9.063353221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:47.136689972 +0000 UTC m=+13.857700256" watchObservedRunningTime="2026-11-03 20:38:47.136988166 +0000 UTC m=+13.857998412"
Nov 03 20:38:47.267782 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:47.267715    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj" podStartSLOduration=1.968931056 podStartE2EDuration="12.267691922s" totalImagesPullingTime="10.298760866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:47.267060605 +0000 UTC m=+13.988070855" watchObservedRunningTime="2026-11-03 20:38:47.267691922 +0000 UTC m=+13.988702169"
Nov 03 20:38:48.018741 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.018636    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-62lhn" event={"ID":"bb233d06-f91f-4db9-be7b-bb9966b67a75","Type":"ContainerStarted","Data":"f7e38e284ee06ac636e64e3587b93722cdaf0d79f82c2327c47a5f966a135b63"}
Nov 03 20:38:48.021217 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.021188    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="0ab97e058bf8a288ebc3a953bd344421dcd7af67d0e4c41caaa4aad087037915" exitCode=0
Nov 03 20:38:48.021339 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.021269    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"0ab97e058bf8a288ebc3a953bd344421dcd7af67d0e4c41caaa4aad087037915"}
Nov 03 20:38:48.137860 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.137799    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-62lhn" podStartSLOduration=0.874294573 podStartE2EDuration="13.137785848s" totalImagesPullingTime="12.263491275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:48.137651078 +0000 UTC m=+14.858661323" watchObservedRunningTime="2026-11-03 20:38:48.137785848 +0000 UTC m=+14.858796093"
Nov 03 20:38:48.821997 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.821970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:48.822193 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.822060    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:48.822193 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:48.822159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:48.822193 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:48.822172    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:48.822352 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:48.822272    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:48.822402 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:48.822365    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:50.028247 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.028003    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="0a782479dafd2cfbc6f375ef2460cf8f96d2ef1aea4234b62e18f832125907db" exitCode=0
Nov 03 20:38:50.028247 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.028095    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"0a782479dafd2cfbc6f375ef2460cf8f96d2ef1aea4234b62e18f832125907db"}
Nov 03 20:38:50.716794 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.716764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:50.716970 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.716815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:50.716970 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.716929    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:50.716970 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.716936    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:50.717097 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.716988    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.716971927 +0000 UTC m=+33.437982161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:50.717097 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.717001    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.71699518 +0000 UTC m=+33.438005400 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:50.817283 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.817253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:50.817467 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.817446    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:50.817527 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.817469    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:50.817527 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.817483    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f46q7 for pod openshift-network-diagnostics/network-check-target-sl28h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:50.817626 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.817549    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7 podName:476f59f4-3347-4e66-b0ea-51add44cfc27 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.817531435 +0000 UTC m=+33.538541659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f46q7" (UniqueName: "kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7") pod "network-check-target-sl28h" (UID: "476f59f4-3347-4e66-b0ea-51add44cfc27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:50.821392 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.821370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:50.821511 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.821397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:50.821511 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.821482    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:50.821511 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:50.821510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:50.821664 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.821593    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:50.821664 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:50.821654    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:52.034737 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:52.034698    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"ed3de395d7e1b6ac66579fe1b0e49b7d646b9a786020b5c465a53acea91d639e"}
Nov 03 20:38:52.035063 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:52.034778    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"f86032b22dfd38b81254e3436c07af18b7bb8160bcdf383fcf6c596cea07be41"}
Nov 03 20:38:52.821436 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:52.821207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:52.821600 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:52.821206    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:52.821600 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:52.821474    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:52.821600 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:52.821207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:52.824151 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:52.822051    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:52.824511 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:52.824486    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:53.038894 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.038861    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="8a236a7de2d972502bed28799ec19e3e7446e400bddd1e11d464c66bfc5ecaab" exitCode=0
Nov 03 20:38:53.039441 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.038938    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"8a236a7de2d972502bed28799ec19e3e7446e400bddd1e11d464c66bfc5ecaab"}
Nov 03 20:38:53.042090 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.042068    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"c2eaedc2149d884721fc9ff52d3ade029a98970065d91458fbebe7172e5d7575"}
Nov 03 20:38:53.042197 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.042148    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"61311ea40f963b06c80d0c32952509d3a49ff5cf4ed8cf3cc48649e355386a21"}
Nov 03 20:38:53.042253 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.042201    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"bbca4ca94b33e502555d6f712ebb96e318e548284b11a2298ee0a29835d53448"}
Nov 03 20:38:53.042253 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:53.042248    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"d210d0eca68b549ad9d12083494721af78dc0068a16bcdfd379c10e96ae8fa07"}
Nov 03 20:38:54.046709 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:54.046680    2578 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="be653610dbc6aedc11034b285763b51780eedeed94507129cdef0ce9618d5c9c" exitCode=0
Nov 03 20:38:54.047130 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:54.046732    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerDied","Data":"be653610dbc6aedc11034b285763b51780eedeed94507129cdef0ce9618d5c9c"}
Nov 03 20:38:54.821632 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:54.821605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:54.821772 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:54.821637    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:54.821772 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:54.821705    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:54.821772 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:54.821620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:54.821772 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:54.821746    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:54.821887 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:54.821798    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:55.051634 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:55.051593    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"fedc06064c9212054cf1f6e4b0ad681339a51b0e6597018498d3a3e3f0852a28"}
Nov 03 20:38:55.054743 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:55.054714    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" event={"ID":"c6d5af76-ea42-41d7-9920-db291e351c53","Type":"ContainerStarted","Data":"02a96c1b82ecd2f0c4eb5bf149ac106539ff17d57fa5195a855878a9a8f9eb73"}
Nov 03 20:38:55.176018 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:55.175933    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vc9ll" podStartSLOduration=6.555549425 podStartE2EDuration="20.175922021s" totalImagesPullingTime="13.620372596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:55.175714283 +0000 UTC m=+21.896724525" watchObservedRunningTime="2026-11-03 20:38:55.175922021 +0000 UTC m=+21.896932263"
Nov 03 20:38:56.821874 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:56.821640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:56.822396 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:56.821640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:56.822396 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:56.821881    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:56.822396 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:56.821948    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:56.822396 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:56.821644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:56.822396 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:56.822031    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:57.062165 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:57.062128    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" event={"ID":"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64","Type":"ContainerStarted","Data":"8cea4df6da3e311888d24b42b67279f0c7631a30f4697dac7ccfdc030d2ba696"}
Nov 03 20:38:57.062369 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:57.062350    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:57.062443 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:57.062427    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:57.079177 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:57.079119    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:57.338446 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:57.338340    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gft95" podStartSLOduration=5.78114171 podStartE2EDuration="22.338328034s" totalImagesPullingTime="16.557186324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:57.337823551 +0000 UTC m=+24.058833794" watchObservedRunningTime="2026-11-03 20:38:57.338328034 +0000 UTC m=+24.059338276"
Nov 03 20:38:58.064025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:58.063995    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:58.079039 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:58.079012    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:38:58.821165 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:58.821135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:58.821165 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:58.821163    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:58.821372 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:58.821252    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:38:58.821372 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:58.821305    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:58.821434 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:58.821399    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:58.821499 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:58.821480    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:59.102097 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.102011    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sl28h"]
Nov 03 20:38:59.102533 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.102108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:38:59.102533 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:59.102200    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:38:59.104985 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.104961    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ffxrp"]
Nov 03 20:38:59.105094 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.105053    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:38:59.105175 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:59.105156    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:38:59.105569 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.105533    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjbt9"]
Nov 03 20:38:59.105672 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:38:59.105642    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:38:59.105891 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:38:59.105870    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:39:00.821833 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:00.821747    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:00.821833 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:00.821766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:00.822377 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:00.821877    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:39:00.822377 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:00.821920    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:00.822377 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:00.821993    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:39:00.822377 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:00.822053    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:39:02.822037 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:02.822007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:02.822407 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:02.822117    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:02.822407 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:02.822127    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:39:02.822407 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:02.822138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:02.822407 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:02.822197    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:39:02.822407 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:02.822240    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:39:04.821477 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:04.821446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:04.821799 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:04.821512    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:04.821799 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:04.821618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:04.821799 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:04.821637    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sl28h" podUID="476f59f4-3347-4e66-b0ea-51add44cfc27"
Nov 03 20:39:04.821799 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:04.821700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xjbt9" podUID="5713f5a0-c8bf-44c1-aeff-c9c580746c45"
Nov 03 20:39:04.821799 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:04.821779    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ffxrp" podUID="b768b17b-8769-4002-8c89-3bee5fda2060"
Nov 03 20:39:05.037422 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.037355    2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeReady"
Nov 03 20:39:05.037539 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.037475    2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 20:39:05.143669 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.143645    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-dnb6d"]
Nov 03 20:39:05.146659 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.146639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:05.151932 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.151908    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l9wgt"]
Nov 03 20:39:05.155383 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.155366    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w4kw4"]
Nov 03 20:39:05.155699 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.155675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:05.158166 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.158141    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zrjt5"]
Nov 03 20:39:05.158288 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.158267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:05.161835 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.161819    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5769c487dc-z9pqs"]
Nov 03 20:39:05.161971 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.161951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.164754 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.164736    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"]
Nov 03 20:39:05.164879 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.164858    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.168142 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.168120    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"]
Nov 03 20:39:05.168245 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.168224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.172017 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.171998    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"]
Nov 03 20:39:05.172249 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.172229    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:05.174714 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.174696    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"]
Nov 03 20:39:05.174839 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.174819    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.177740 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.177719    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"]
Nov 03 20:39:05.178610 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.177850    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.185983 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.185961    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"]
Nov 03 20:39:05.186102 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.186084    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.189440 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.189184    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"]
Nov 03 20:39:05.189655 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.189542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.191948 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.191930    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:39:05.192054 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.192032    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.194931 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.194911    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:39:05.195048 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.195023    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.197996 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.197979    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"]
Nov 03 20:39:05.198114 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.198092    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.200856 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.200833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:05.201241 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.201226    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zjglm"]
Nov 03 20:39:05.205250 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.204257    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.205250 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.204354    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.205641 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.205625    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6sv74"]
Nov 03 20:39:05.205768 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.205750    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.208430 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.208411    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-dnb6d"]
Nov 03 20:39:05.208508 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.208444    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l9wgt"]
Nov 03 20:39:05.208572 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.208532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.220550 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.220529    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.220651 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.220581    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.221116 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.221093    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 03 20:39:05.225863 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.225847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.225956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.225873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.225956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.225889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.225956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.225905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.225956 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.225927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47h9w\" (UniqueName: \"kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w\") pod \"kas-connection-checker-685895c7c9-l9wgt\" (UID: \"90155d8a-eff2-4b22-ac1b-975bffe1d644\") " pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:05.226136 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226029    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlhzq\" (UniqueName: \"kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq\") pod \"kas-connection-checker-685895c7c9-w4kw4\" (UID: \"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb\") " pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:05.226136 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226093    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.226203 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.226203 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226195    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cr8r\" (UniqueName: \"kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.226273 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.226273 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxkq8\" (UniqueName: \"kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.226370 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.226410 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72v9b\" (UniqueName: \"kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b\") pod \"kas-connection-checker-685895c7c9-dnb6d\" (UID: \"30b966c5-d3d0-4743-a041-fe5f0f196fd1\") " pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:05.226410 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.226398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.253072 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.253052    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 03 20:39:05.253800 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.253783    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 03 20:39:05.254905 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.254884    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zrjt5"]
Nov 03 20:39:05.271318 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.271297    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5769c487dc-z9pqs"]
Nov 03 20:39:05.272385 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.272309    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"]
Nov 03 20:39:05.273369 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.273350    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"]
Nov 03 20:39:05.274511 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.274493    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"]
Nov 03 20:39:05.275483 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.275462    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"]
Nov 03 20:39:05.276335 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.276316    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"]
Nov 03 20:39:05.277302 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.277281    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"]
Nov 03 20:39:05.278214 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.278195    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"]
Nov 03 20:39:05.280484 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.280463    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:39:05.285729 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.284242    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w4kw4"]
Nov 03 20:39:05.285729 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.285478    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zjglm"]
Nov 03 20:39:05.302945 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.302818    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"]
Nov 03 20:39:05.303393 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303364    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.303546 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303516    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.303672 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303605    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.303672 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303668    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303605    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303705    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303785    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303809    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303832    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303857    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303877    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303897    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303916    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.303936    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304004    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304246    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304269    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304292    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.304320 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304338    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304350    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304371    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304381    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304393    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304417    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.304632    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 03 20:39:05.305270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.305055    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 03 20:39:05.326774 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326738    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xpb\" (UniqueName: \"kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.326875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.326932 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.326932 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.326932 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cbl\" (UniqueName: \"kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.327085 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.327085 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.327085 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.326996    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.327220 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.327220 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.327220 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.327220 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327237    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9s5\" (UniqueName: \"kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.327288    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.827268954 +0000 UTC m=+32.548279188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : configmap references non-existent config key: service-ca.crt
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.327403 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlhzq\" (UniqueName: \"kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq\") pod \"kas-connection-checker-685895c7c9-w4kw4\" (UID: \"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb\") " pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327447    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8qts\" (UniqueName: \"kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ggk7\" (UniqueName: \"kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxkq8\" (UniqueName: \"kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.327752 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrl4g\" (UniqueName: \"kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwzrq\" (UniqueName: \"kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327949    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.327999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.328069 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72v9b\" (UniqueName: \"kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b\") pod \"kas-connection-checker-685895c7c9-dnb6d\" (UID: \"30b966c5-d3d0-4743-a041-fe5f0f196fd1\") " pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrs8\" (UniqueName: \"kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8\") pod \"volume-data-source-validator-58c9f5f46d-v6ssz\" (UID: \"6c893222-ab83-4a89-98e2-169613906e67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:05.328316 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328317    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.328591 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328571    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47h9w\" (UniqueName: \"kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w\") pod \"kas-connection-checker-685895c7c9-l9wgt\" (UID: \"90155d8a-eff2-4b22-ac1b-975bffe1d644\") " pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328745    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqjr\" (UniqueName: \"kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr\") pod \"network-check-source-6f78d59f99-f6gzh\" (UID: \"54725514-9087-42f7-8c47-5a995d71a0ee\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.328866 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.329154 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.328989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.329154 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.329047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cr8r\" (UniqueName: \"kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.329154 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.329091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.329674 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.329629    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.330197 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.330122    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.330965 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.330942    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.331083 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.330993    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.332830 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.332811    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 03 20:39:05.333004 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.332982    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 03 20:39:05.334827 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.334804    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.351612 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351586    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 03 20:39:05.351612 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351596    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 03 20:39:05.351786 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351653    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 03 20:39:05.351786 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351663    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 03 20:39:05.351786 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351663    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 03 20:39:05.351786 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351663    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-plrnn\""
Nov 03 20:39:05.351786 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.351688    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 03 20:39:05.352538 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352521    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 20:39:05.352803 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352789    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 03 20:39:05.352875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352810    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 03 20:39:05.352875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352807    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 03 20:39:05.352875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352829    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 03 20:39:05.352875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352843    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 20:39:05.352875 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352856    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 03 20:39:05.353131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352883    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9ll5z\""
Nov 03 20:39:05.353131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352914    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fjpbj\""
Nov 03 20:39:05.353131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.352914    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 03 20:39:05.354016 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.354002    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:39:05.354595 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.354582    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6sv74"]
Nov 03 20:39:05.357177 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.357158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:05.358055 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.358032    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 20:39:05.358164 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.358096    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.858083167 +0000 UTC m=+32.579093421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : secret "router-metrics-certs-default" not found
Nov 03 20:39:05.360296 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.360273    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.360386 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.360354    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.390161 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.390133    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxkq8\" (UniqueName: \"kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.390335 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.390305    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlhzq\" (UniqueName: \"kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq\") pod \"kas-connection-checker-685895c7c9-w4kw4\" (UID: \"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb\") " pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:05.390516 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.390495    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72v9b\" (UniqueName: \"kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b\") pod \"kas-connection-checker-685895c7c9-dnb6d\" (UID: \"30b966c5-d3d0-4743-a041-fe5f0f196fd1\") " pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:05.391876 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.391853    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47h9w\" (UniqueName: \"kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w\") pod \"kas-connection-checker-685895c7c9-l9wgt\" (UID: \"90155d8a-eff2-4b22-ac1b-975bffe1d644\") " pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:05.392397 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.392382    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cr8r\" (UniqueName: \"kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.430468 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430447    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.430575 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9s5\" (UniqueName: \"kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.430575 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.430575 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.430575 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.430716 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.430716 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.430805 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.430734    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:05.430805 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.430750    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98474cc5c-vc78r: secret "image-registry-tls" not found
Nov 03 20:39:05.430889 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.430806    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls podName:0f528bfd-d20d-4fe1-81ae-754987ca453f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.93078537 +0000 UTC m=+32.651795608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls") pod "image-registry-98474cc5c-vc78r" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f") : secret "image-registry-tls" not found
Nov 03 20:39:05.430889 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.430889 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.430883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8qts\" (UniqueName: \"kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.431041 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ggk7\" (UniqueName: \"kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.431094 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.431094 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431081    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrl4g\" (UniqueName: \"kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.431213 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.431328 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431302    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.431386 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431342    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.431605 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwzrq\" (UniqueName: \"kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.431700 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.431771 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.431771 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431596    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.431771 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.431923 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.431975 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431925    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.432032 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.431991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.432032 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.432131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrs8\" (UniqueName: \"kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8\") pod \"volume-data-source-validator-58c9f5f46d-v6ssz\" (UID: \"6c893222-ab83-4a89-98e2-169613906e67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:05.432131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.432131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.432292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.432292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.432292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.432292 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432296    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjr\" (UniqueName: \"kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr\") pod \"network-check-source-6f78d59f99-f6gzh\" (UID: \"54725514-9087-42f7-8c47-5a995d71a0ee\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432342    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.432390    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.432404    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf6c99f88-pj2bd: secret "image-registry-tls" not found
Nov 03 20:39:05.432487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.432453    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls podName:1747a6b1-0e38-4948-baa2-a75cbe265752 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.932435232 +0000 UTC m=+32.653445453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls") pod "image-registry-5cf6c99f88-pj2bd" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752") : secret "image-registry-tls" not found
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.432952    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.433011    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.932996034 +0000 UTC m=+32.654006270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433146    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433277    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433293    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.433374    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.433425    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.93341005 +0000 UTC m=+32.654420285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433510    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433516    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.432388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433654    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xpb\" (UniqueName: \"kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.433928 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433775    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cbl\" (UniqueName: \"kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.434005    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.434053    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.934040056 +0000 UTC m=+32.655050297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.434072    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.433800    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.434453 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.434119    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:05.934107491 +0000 UTC m=+32.655117726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:05.434993 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.434872    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.434993 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.434892    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.435503 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.435478    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.435628 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.435507    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.435783 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.435762    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.435855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.435834    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.436107 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.436089    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.436745 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.436726    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.436892 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.436877    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.436939 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.436916    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.461251 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.461232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:05.468877 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.468854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:05.476534 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.476518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:05.485547 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.485523    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:05.495167 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.495121    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.497694 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.497643    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9s5\" (UniqueName: \"kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.503623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.500156    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrl4g\" (UniqueName: \"kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.503623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.500370    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8qts\" (UniqueName: \"kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.503623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.500449    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ggk7\" (UniqueName: \"kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.503623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.502790    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cbl\" (UniqueName: \"kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.503882 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.503670    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.504709 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.504645    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqjr\" (UniqueName: \"kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr\") pod \"network-check-source-6f78d59f99-f6gzh\" (UID: \"54725514-9087-42f7-8c47-5a995d71a0ee\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:05.504822 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.504794    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrs8\" (UniqueName: \"kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8\") pod \"volume-data-source-validator-58c9f5f46d-v6ssz\" (UID: \"6c893222-ab83-4a89-98e2-169613906e67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:05.505183 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.505154    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.506229 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.506171    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xpb\" (UniqueName: \"kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.507383 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.507329    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.507487 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.507416    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwzrq\" (UniqueName: \"kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.510347 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.510207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:05.521811 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.521786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:05.532319 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.529708    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:05.534277 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.534259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:05.560526 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.560306    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:05.564957 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.564713    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:05.599230 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.579042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:05.839953 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.839886    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.840270 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.840018    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.840005615 +0000 UTC m=+33.561015837 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : configmap references non-existent config key: service-ca.crt
Nov 03 20:39:05.861644 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.861610    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-l9wgt"]
Nov 03 20:39:05.865059 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.865036    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90155d8a_eff2_4b22_ac1b_975bffe1d644.slice/crio-4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340 WatchSource:0}: Error finding container 4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340: Status 404 returned error can't find the container with id 4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340
Nov 03 20:39:05.893370 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.893344    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-w4kw4"]
Nov 03 20:39:05.900585 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.899104    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5e0dcf_8842_4b4a_bc4b_d11e653865fb.slice/crio-5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0 WatchSource:0}: Error finding container 5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0: Status 404 returned error can't find the container with id 5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0
Nov 03 20:39:05.921103 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.921072    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-dnb6d"]
Nov 03 20:39:05.925941 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.925913    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30b966c5_d3d0_4743_a041_fe5f0f196fd1.slice/crio-46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b WatchSource:0}: Error finding container 46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b: Status 404 returned error can't find the container with id 46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b
Nov 03 20:39:05.940760 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.940730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:05.940871 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.940791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:05.940871 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.940859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:05.940997 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.940972    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:05.941062 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941032    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:05.941062 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.940975    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:05.941062 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941057    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf6c99f88-pj2bd: secret "image-registry-tls" not found
Nov 03 20:39:05.941158 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941036    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941017155 +0000 UTC m=+33.662027397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:05.941158 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941100    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941084586 +0000 UTC m=+33.662094808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:05.941158 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.941133    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:05.941255 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.941167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:05.941255 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.941199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:05.941309 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.941269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:05.941393 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941377    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls podName:1747a6b1-0e38-4948-baa2-a75cbe265752 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941360675 +0000 UTC m=+33.662370896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls") pod "image-registry-5cf6c99f88-pj2bd" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752") : secret "image-registry-tls" not found
Nov 03 20:39:05.941487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941399    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 20:39:05.941487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941415    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:05.941487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941433    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941421802 +0000 UTC m=+33.662432048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : secret "router-metrics-certs-default" not found
Nov 03 20:39:05.941487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941447    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941438124 +0000 UTC m=+33.662448345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:05.941487 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941476    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:05.941704 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941490    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98474cc5c-vc78r: secret "image-registry-tls" not found
Nov 03 20:39:05.941704 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941528    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls podName:0f528bfd-d20d-4fe1-81ae-754987ca453f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941514528 +0000 UTC m=+33.662524751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls") pod "image-registry-98474cc5c-vc78r" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f") : secret "image-registry-tls" not found
Nov 03 20:39:05.941704 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941586    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:05.941704 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:05.941633    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:06.941620797 +0000 UTC m=+33.662631051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:05.944520 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.944497    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-zrjt5"]
Nov 03 20:39:05.950364 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.950341    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c22090d_43fa_4fa7_adac_ecff9757f988.slice/crio-2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f WatchSource:0}: Error finding container 2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f: Status 404 returned error can't find the container with id 2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f
Nov 03 20:39:05.967611 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.967525    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"]
Nov 03 20:39:05.989206 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:05.989181    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"]
Nov 03 20:39:05.994218 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.994177    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c893222_ab83_4a89_98e2_169613906e67.slice/crio-f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152 WatchSource:0}: Error finding container f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152: Status 404 returned error can't find the container with id f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152
Nov 03 20:39:05.996685 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:05.996647    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54725514_9087_42f7_8c47_5a995d71a0ee.slice/crio-fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2 WatchSource:0}: Error finding container fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2: Status 404 returned error can't find the container with id fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2
Nov 03 20:39:06.006281 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.006257    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"]
Nov 03 20:39:06.009508 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:06.009484    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab0970d_6322_42d0_a66b_6d0acba24275.slice/crio-b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6 WatchSource:0}: Error finding container b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6: Status 404 returned error can't find the container with id b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6
Nov 03 20:39:06.019101 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.019057    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"]
Nov 03 20:39:06.023065 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.022355    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"]
Nov 03 20:39:06.026983 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:06.025952    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d37aa0c_7893_407a_a52a_46d8a1ffd00e.slice/crio-67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4 WatchSource:0}: Error finding container 67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4: Status 404 returned error can't find the container with id 67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4
Nov 03 20:39:06.054442 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.054422    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6sv74"]
Nov 03 20:39:06.057449 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:06.057431    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91453c10_6f59_4fd5_bf2a_69c2c3944375.slice/crio-c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd WatchSource:0}: Error finding container c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd: Status 404 returned error can't find the container with id c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd
Nov 03 20:39:06.079666 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.079643    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz" event={"ID":"6c893222-ab83-4a89-98e2-169613906e67","Type":"ContainerStarted","Data":"f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152"}
Nov 03 20:39:06.081043 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.081021    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-dnb6d" event={"ID":"30b966c5-d3d0-4743-a041-fe5f0f196fd1","Type":"ContainerStarted","Data":"a50efab8d0ed5f490ce406c22dced93a9eccfe41b74b2ddbcee86e2eb83c980b"}
Nov 03 20:39:06.081136 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.081065    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-dnb6d" event={"ID":"30b966c5-d3d0-4743-a041-fe5f0f196fd1","Type":"ContainerStarted","Data":"46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b"}
Nov 03 20:39:06.082212 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.082194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh" event={"ID":"54725514-9087-42f7-8c47-5a995d71a0ee","Type":"ContainerStarted","Data":"fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2"}
Nov 03 20:39:06.083302 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.083281    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx" event={"ID":"bab0970d-6322-42d0-a66b-6d0acba24275","Type":"ContainerStarted","Data":"b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6"}
Nov 03 20:39:06.084687 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.084661    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-w4kw4" event={"ID":"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb","Type":"ContainerStarted","Data":"0673915c565817a6f746e96d0418f57b181abb1892ce6fa557a3243011c04d9e"}
Nov 03 20:39:06.084749 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.084699    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-w4kw4" event={"ID":"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb","Type":"ContainerStarted","Data":"5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0"}
Nov 03 20:39:06.085812 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.085792    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6sv74" event={"ID":"91453c10-6f59-4fd5-bf2a-69c2c3944375","Type":"ContainerStarted","Data":"c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd"}
Nov 03 20:39:06.086894 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.086868    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" event={"ID":"6d37aa0c-7893-407a-a52a-46d8a1ffd00e","Type":"ContainerStarted","Data":"67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4"}
Nov 03 20:39:06.087882 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.087865    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph" event={"ID":"83806e59-a55b-4491-b6d0-59b373bac613","Type":"ContainerStarted","Data":"5c92695d378a43c8d841ce5f8f23e24c7e07287b695ec1a9aa2c19b3019e3a45"}
Nov 03 20:39:06.088950 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.088933    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zrjt5" event={"ID":"1c22090d-43fa-4fa7-adac-ecff9757f988","Type":"ContainerStarted","Data":"2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f"}
Nov 03 20:39:06.090321 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.090280    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-l9wgt" event={"ID":"90155d8a-eff2-4b22-ac1b-975bffe1d644","Type":"ContainerStarted","Data":"b5e94e1458263636ed77959d012c987c1605bbe2a7143d4c8fa9736433bd0c3e"}
Nov 03 20:39:06.090321 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.090308    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-l9wgt" event={"ID":"90155d8a-eff2-4b22-ac1b-975bffe1d644","Type":"ContainerStarted","Data":"4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340"}
Nov 03 20:39:06.747876 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.747846    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:06.747876 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.747892    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:06.748085 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.747989    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:06.748085 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.748002    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:06.748085 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.748046    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs podName:b768b17b-8769-4002-8c89-3bee5fda2060 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:38.748034333 +0000 UTC m=+65.469044555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs") pod "network-metrics-daemon-ffxrp" (UID: "b768b17b-8769-4002-8c89-3bee5fda2060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:06.748085 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.748057    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret podName:5713f5a0-c8bf-44c1-aeff-c9c580746c45 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:38.748051827 +0000 UTC m=+65.469062049 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret") pod "global-pull-secret-syncer-xjbt9" (UID: "5713f5a0-c8bf-44c1-aeff-c9c580746c45") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:06.821870 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.821833    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:06.821870 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.821864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:06.822061 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.821834    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:06.848221 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.848198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:06.848500 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.848259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:06.848500 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.848406    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.848386191 +0000 UTC m=+35.569396413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : configmap references non-existent config key: service-ca.crt
Nov 03 20:39:06.850692 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.850668    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:06.876243 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.876216    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:06.876354 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.876251    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:06.876354 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.876263    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:06.876354 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.876285    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:06.876354 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.876291    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zvv9n\""
Nov 03 20:39:06.948830 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:06.949005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:06.949005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:06.949005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:06.949005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:06.949005 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.948987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:06.949255 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:06.949087    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:06.949327 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949294    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 20:39:06.949379 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949364    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949347955 +0000 UTC m=+35.670358192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : secret "router-metrics-certs-default" not found
Nov 03 20:39:06.949482 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949468    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:06.949482 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949469    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:06.949631 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949490    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf6c99f88-pj2bd: secret "image-registry-tls" not found
Nov 03 20:39:06.949631 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949496    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949489031 +0000 UTC m=+35.670499265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:06.949631 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949581    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls podName:1747a6b1-0e38-4948-baa2-a75cbe265752 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949541972 +0000 UTC m=+35.670552211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls") pod "image-registry-5cf6c99f88-pj2bd" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752") : secret "image-registry-tls" not found
Nov 03 20:39:06.949772 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949697    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:06.949772 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949707    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98474cc5c-vc78r: secret "image-registry-tls" not found
Nov 03 20:39:06.949772 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949740    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls podName:0f528bfd-d20d-4fe1-81ae-754987ca453f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949728411 +0000 UTC m=+35.670738633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls") pod "image-registry-98474cc5c-vc78r" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f") : secret "image-registry-tls" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949788    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949793    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949811    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949824    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949813263 +0000 UTC m=+35.670823503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949848    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949832229 +0000 UTC m=+35.670842477 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:06.949892 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:06.949864    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:08.949857668 +0000 UTC m=+35.670867888 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:07.135645 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:07.135304    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:07.271435 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:07.271266    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-l9wgt" podStartSLOduration=379.271222638 podStartE2EDuration="6m19.271222638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:07.269092257 +0000 UTC m=+33.990102504" watchObservedRunningTime="2026-11-03 20:39:07.271222638 +0000 UTC m=+33.992232878"
Nov 03 20:39:07.505476 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:07.505168    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-w4kw4" podStartSLOduration=379.50515188 podStartE2EDuration="6m19.50515188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:07.390085951 +0000 UTC m=+34.111096186" watchObservedRunningTime="2026-11-03 20:39:07.50515188 +0000 UTC m=+34.226162124"
Nov 03 20:39:07.567025 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:07.566962    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-dnb6d" podStartSLOduration=379.566942177 podStartE2EDuration="6m19.566942177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:07.5066963 +0000 UTC m=+34.227706549" watchObservedRunningTime="2026-11-03 20:39:07.566942177 +0000 UTC m=+34.287952423"
Nov 03 20:39:07.567196 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:07.567114    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sl28h"]
Nov 03 20:39:07.583384 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:07.583349    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476f59f4_3347_4e66_b0ea_51add44cfc27.slice/crio-bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b WatchSource:0}: Error finding container bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b: Status 404 returned error can't find the container with id bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b
Nov 03 20:39:08.107781 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.107696    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sl28h" event={"ID":"476f59f4-3347-4e66-b0ea-51add44cfc27","Type":"ContainerStarted","Data":"bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b"}
Nov 03 20:39:08.748006 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.747966    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:08.869523 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.868746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:08.869523 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.869096    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.869077757 +0000 UTC m=+39.590087984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : configmap references non-existent config key: service-ca.crt
Nov 03 20:39:08.896000 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.895909    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:08.970440 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.970363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:08.970661 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.970472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:08.970725 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.970700    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 20:39:08.970784 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.970767    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.970747674 +0000 UTC m=+39.691757900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : secret "router-metrics-certs-default" not found
Nov 03 20:39:08.970880 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.970844    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:08.970880 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.970858    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98474cc5c-vc78r: secret "image-registry-tls" not found
Nov 03 20:39:08.970978 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.970892    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls podName:0f528bfd-d20d-4fe1-81ae-754987ca453f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.970881445 +0000 UTC m=+39.691891669 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls") pod "image-registry-98474cc5c-vc78r" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f") : secret "image-registry-tls" not found
Nov 03 20:39:08.970978 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.970552    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:08.971076 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.970986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:08.971076 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.971021    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:08.971076 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.971065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:08.971227 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:08.971096    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:08.971227 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971197    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:08.971322 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971239    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.971228848 +0000 UTC m=+39.692239071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:08.971322 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971294    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:08.971322 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971304    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf6c99f88-pj2bd: secret "image-registry-tls" not found
Nov 03 20:39:08.971531 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971337    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls podName:1747a6b1-0e38-4948-baa2-a75cbe265752 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.971321025 +0000 UTC m=+39.692331251 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls") pod "image-registry-5cf6c99f88-pj2bd" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752") : secret "image-registry-tls" not found
Nov 03 20:39:08.971531 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971390    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:08.971531 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971423    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.971413376 +0000 UTC m=+39.692423599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:08.971531 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971472    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:08.971531 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971513    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.971502685 +0000 UTC m=+39.692512907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:08.971826 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971595    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:08.971826 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:08.971625    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:12.971615598 +0000 UTC m=+39.692625828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:09.134529 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:09.134492    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz" event={"ID":"6c893222-ab83-4a89-98e2-169613906e67","Type":"ContainerStarted","Data":"bfe0346e6b82850eff1bf70f07dd662d08bd867d97808c18c26906c5a2c4a587"}
Nov 03 20:39:09.359543 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:09.359454    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:09.517595 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:09.517541    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:10.195785 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:10.195749    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:11.145847 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:11.145818    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/0.log"
Nov 03 20:39:11.146018 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:11.145859    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" containerID="c7c927c9f632e0efbfb95e343737f39dcb3af95818d46635e070308f4b2562a5" exitCode=255
Nov 03 20:39:11.146018 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:11.145934    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" event={"ID":"6d37aa0c-7893-407a-a52a-46d8a1ffd00e","Type":"ContainerDied","Data":"c7c927c9f632e0efbfb95e343737f39dcb3af95818d46635e070308f4b2562a5"}
Nov 03 20:39:11.146340 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:11.146325    2578 scope.go:122] "RemoveContainer" containerID="c7c927c9f632e0efbfb95e343737f39dcb3af95818d46635e070308f4b2562a5"
Nov 03 20:39:11.264755 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:11.264621    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz" podStartSLOduration=348.495397208 podStartE2EDuration="5m51.264598953s" totalImagesPullingTime="2.769201745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:09.250966037 +0000 UTC m=+35.971976281" watchObservedRunningTime="2026-11-03 20:39:11.264598953 +0000 UTC m=+37.985609197"
Nov 03 20:39:12.152095 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.152055    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx" event={"ID":"bab0970d-6322-42d0-a66b-6d0acba24275","Type":"ContainerStarted","Data":"5ae1bcca9392072495d748bf0bea39324a2809632ed3bf1b41f19c7f1777f95f"}
Nov 03 20:39:12.156120 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156098    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/1.log"
Nov 03 20:39:12.156598 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156576    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/0.log"
Nov 03 20:39:12.156700 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156623    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478" exitCode=255
Nov 03 20:39:12.156700 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156674    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" event={"ID":"6d37aa0c-7893-407a-a52a-46d8a1ffd00e","Type":"ContainerDied","Data":"f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"}
Nov 03 20:39:12.156805 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156719    2578 scope.go:122] "RemoveContainer" containerID="c7c927c9f632e0efbfb95e343737f39dcb3af95818d46635e070308f4b2562a5"
Nov 03 20:39:12.156911 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.156894    2578 scope.go:122] "RemoveContainer" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"
Nov 03 20:39:12.157073 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:12.157055    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-xv2zb_openshift-console-operator(6d37aa0c-7893-407a-a52a-46d8a1ffd00e)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" podUID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e"
Nov 03 20:39:12.158838 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.158820    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph" event={"ID":"83806e59-a55b-4491-b6d0-59b373bac613","Type":"ContainerStarted","Data":"9eda50cde32c01dbef48fdaa9377dbe4f560e528f9c6345e47b3d427c6540f92"}
Nov 03 20:39:12.160643 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.160398    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zrjt5" event={"ID":"1c22090d-43fa-4fa7-adac-ecff9757f988","Type":"ContainerStarted","Data":"9d5effa72a0949b1aacdedb1d8d3d75f7ec0c1c9370f21fd0c0e13793739cc06"}
Nov 03 20:39:12.270485 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.270428    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx" podStartSLOduration=346.722139207 podStartE2EDuration="5m52.270413474s" totalImagesPullingTime="5.548274267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:12.268182464 +0000 UTC m=+38.989192706" watchObservedRunningTime="2026-11-03 20:39:12.270413474 +0000 UTC m=+38.991423715"
Nov 03 20:39:12.384280 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.384210    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph" podStartSLOduration=346.877197069 podStartE2EDuration="5m52.384188102s" totalImagesPullingTime="5.506991033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:12.38288433 +0000 UTC m=+39.103894586" watchObservedRunningTime="2026-11-03 20:39:12.384188102 +0000 UTC m=+39.105198347"
Nov 03 20:39:12.617073 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.617010    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-zrjt5" podStartSLOduration=347.033592192 podStartE2EDuration="5m52.616996845s" totalImagesPullingTime="5.583404653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:12.61630417 +0000 UTC m=+39.337314413" watchObservedRunningTime="2026-11-03 20:39:12.616996845 +0000 UTC m=+39.338007111"
Nov 03 20:39:12.922665 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:12.922585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:12.922826 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:12.922762    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:20.922741243 +0000 UTC m=+47.643751466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : configmap references non-existent config key: service-ca.crt
Nov 03 20:39:13.023083 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:13.023279 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023130    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:13.023279 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:13.023279 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023211    2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023287    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs podName:4c2480e6-eaa3-402a-b3fc-2c63f26ff172 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023267419 +0000 UTC m=+47.744277642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs") pod "router-default-5769c487dc-z9pqs" (UID: "4c2480e6-eaa3-402a-b3fc-2c63f26ff172") : secret "router-metrics-certs-default" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023292    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023306    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf6c99f88-pj2bd: secret "image-registry-tls" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023361    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls podName:1747a6b1-0e38-4948-baa2-a75cbe265752 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023346335 +0000 UTC m=+47.744356562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls") pod "image-registry-5cf6c99f88-pj2bd" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752") : secret "image-registry-tls" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023403    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:13.023431 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023447    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023434641 +0000 UTC m=+47.744444877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.023465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023487    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023518    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023507972 +0000 UTC m=+47.744518200 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023291    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023532    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-98474cc5c-vc78r: secret "image-registry-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023573    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls podName:0f528bfd-d20d-4fe1-81ae-754987ca453f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023548127 +0000 UTC m=+47.744558354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls") pod "image-registry-98474cc5c-vc78r" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f") : secret "image-registry-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023623    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023627    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023666    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023655662 +0000 UTC m=+47.744665889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:13.023839 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.023680    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:21.023671738 +0000 UTC m=+47.744681990 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:13.165270 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.165237    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6sv74" event={"ID":"91453c10-6f59-4fd5-bf2a-69c2c3944375","Type":"ContainerStarted","Data":"dc9534af109a48f87c1958800b1ece621e098c43e37c8104d61608f71e4e3ab5"}
Nov 03 20:39:13.166796 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.166773    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/1.log"
Nov 03 20:39:13.167343 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.167120    2578 scope.go:122] "RemoveContainer" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"
Nov 03 20:39:13.167343 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.167304    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-xv2zb_openshift-console-operator(6d37aa0c-7893-407a-a52a-46d8a1ffd00e)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" podUID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e"
Nov 03 20:39:13.168419 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.168387    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh" event={"ID":"54725514-9087-42f7-8c47-5a995d71a0ee","Type":"ContainerStarted","Data":"83f17c7d6320ffa007ef57c353959f01d97e0c606cce8fe6b506f41770a323ac"}
Nov 03 20:39:13.205825 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.205770    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tph4g"]
Nov 03 20:39:13.217997 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.217979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.269368 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.269345    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tph4g"]
Nov 03 20:39:13.270516 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.270493    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 20:39:13.270855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.270526    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 20:39:13.270855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.270584    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fj5k\""
Nov 03 20:39:13.303569 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.303521    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6sv74" podStartSLOduration=2.332406325 podStartE2EDuration="8.303503398s" totalImagesPullingTime="5.971097073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:13.30259672 +0000 UTC m=+40.023606976" watchObservedRunningTime="2026-11-03 20:39:13.303503398 +0000 UTC m=+40.024513640"
Nov 03 20:39:13.326679 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.326637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.327193 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.327033    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp5qz\" (UniqueName: \"kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.327306 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.327206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.327306 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.327264    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.327431 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.327411    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.427835 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.427814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.427953 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.427864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp5qz\" (UniqueName: \"kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.427953 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.427934    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.428056 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.428036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.428118 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.428103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.428180 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.428165    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:13.428212 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.428202    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 20:39:13.428241 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.428168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.428271 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.428240    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:13.928221166 +0000 UTC m=+40.649231405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:13.428325 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.428282    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:13.928259308 +0000 UTC m=+40.649269543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : configmap "kube-rbac-proxy" not found
Nov 03 20:39:13.428463 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.428447    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.432696 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.432667    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:13.460348 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.460288    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:39:13.488374 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.488354    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp5qz\" (UniqueName: \"kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.490081 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.490049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.525491 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.525466    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:39:13.542191 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.542170    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vnlw8\""
Nov 03 20:39:13.586496 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.586466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:13.630310 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.630234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.630445 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.630364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.630445 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.630387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.630703 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.630594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.655466 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.655229    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh" podStartSLOduration=344.128135063 podStartE2EDuration="5m50.655214341s" totalImagesPullingTime="6.527079278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:13.654548448 +0000 UTC m=+40.375558692" watchObservedRunningTime="2026-11-03 20:39:13.655214341 +0000 UTC m=+40.376224587"
Nov 03 20:39:13.698508 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.698481    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"]
Nov 03 20:39:13.724167 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.724105    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"]
Nov 03 20:39:13.724286 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.724218    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:13.731234 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.731338 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.731338 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.731468 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731454    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.731625 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731606    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.731947 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.731931    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.732051 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.732034    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.746428 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.746408    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:13.763544 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.763518    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"]
Nov 03 20:39:13.764815 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.764793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:13.776819 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.776791    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"]
Nov 03 20:39:13.777663 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.777641    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:13.777811 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.777794    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 03 20:39:13.778694 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.778681    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2v8j5\""
Nov 03 20:39:13.792608 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.792587    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.820530 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.820510    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:13.832613 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.832590    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj8wv\" (UniqueName: \"kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv\") pod \"migrator-66df5d496b-wjr2w\" (UID: \"8e590e6f-ebec-4c39-8f00-d9460e3aeb57\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:13.933720 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.933683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.933720 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.933718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8wv\" (UniqueName: \"kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv\") pod \"migrator-66df5d496b-wjr2w\" (UID: \"8e590e6f-ebec-4c39-8f00-d9460e3aeb57\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:13.933901 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.933845    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:13.933901 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.933863    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:13.933962 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.933922    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:14.933902566 +0000 UTC m=+41.654912788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:13.933962 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.933942    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 20:39:13.934034 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:13.934002    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:14.933990288 +0000 UTC m=+41.655000509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : configmap "kube-rbac-proxy" not found
Nov 03 20:39:13.934076 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.934050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssmxp\" (UniqueName: \"kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp\") pod \"migrator-66df5d496b-9zp4f\" (UID: \"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:13.943648 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:13.943621    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:14.000686 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.000629    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8wv\" (UniqueName: \"kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv\") pod \"migrator-66df5d496b-wjr2w\" (UID: \"8e590e6f-ebec-4c39-8f00-d9460e3aeb57\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:14.035395 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.035367    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssmxp\" (UniqueName: \"kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp\") pod \"migrator-66df5d496b-9zp4f\" (UID: \"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:14.056733 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.056707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:14.104499 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.104471    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssmxp\" (UniqueName: \"kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp\") pod \"migrator-66df5d496b-9zp4f\" (UID: \"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:14.128671 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.128626    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:14.173030 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.172985    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sl28h" event={"ID":"476f59f4-3347-4e66-b0ea-51add44cfc27","Type":"ContainerStarted","Data":"7cc4349cb9b8bb841fa46a334ee1d0c9c83464e6b29c0ee3ded5babbac8af151"}
Nov 03 20:39:14.207058 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.207037    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:39:14.209230 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:14.209201    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1f9441_f48b_4ba2_bfc3_e48b69ec5d32.slice/crio-e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224 WatchSource:0}: Error finding container e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224: Status 404 returned error can't find the container with id e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224
Nov 03 20:39:14.275331 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.275279    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:14.289409 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.289360    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sl28h" podStartSLOduration=34.494888246 podStartE2EDuration="40.28934344s" totalImagesPullingTime="5.794455194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:14.287891303 +0000 UTC m=+41.008901545" watchObservedRunningTime="2026-11-03 20:39:14.28934344 +0000 UTC m=+41.010353683"
Nov 03 20:39:14.378422 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.378383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:14.466529 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.466353    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"]
Nov 03 20:39:14.469096 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:14.469065    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e590e6f_ebec_4c39_8f00_d9460e3aeb57.slice/crio-99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a WatchSource:0}: Error finding container 99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a: Status 404 returned error can't find the container with id 99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a
Nov 03 20:39:14.764552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.764479    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"]
Nov 03 20:39:14.767521 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:14.767498    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78df5c50_3dd5_4a0e_a01e_8f790b7dd1fe.slice/crio-40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8 WatchSource:0}: Error finding container 40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8: Status 404 returned error can't find the container with id 40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8
Nov 03 20:39:14.944081 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.944046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:14.944236 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.944098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:14.944236 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:14.944164    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 20:39:14.944236 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:14.944195    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:14.944236 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:14.944225    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:16.944208854 +0000 UTC m=+43.665219099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : configmap "kube-rbac-proxy" not found
Nov 03 20:39:14.944369 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:14.944246    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:16.944233035 +0000 UTC m=+43.665243256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:14.947702 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.947680    2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-r2rmg"]
Nov 03 20:39:14.979131 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:14.979109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.011163 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.011134    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-r2rmg"]
Nov 03 20:39:15.032914 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.032855    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 03 20:39:15.032914 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.032903    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 03 20:39:15.033078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.032939    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 03 20:39:15.033078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.032977    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p5s6h\""
Nov 03 20:39:15.033078 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.032998    2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 03 20:39:15.146472 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.146444    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djxhv\" (UniqueName: \"kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.146659 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.146602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.146659 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.146634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.151229 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.151206    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:15.176936 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.176907    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f" event={"ID":"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe","Type":"ContainerStarted","Data":"40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8"}
Nov 03 20:39:15.178174 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.178153    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w" event={"ID":"8e590e6f-ebec-4c39-8f00-d9460e3aeb57","Type":"ContainerStarted","Data":"99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a"}
Nov 03 20:39:15.179696 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.179674    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" event={"ID":"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32","Type":"ContainerStarted","Data":"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"}
Nov 03 20:39:15.179796 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.179709    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" event={"ID":"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32","Type":"ContainerStarted","Data":"e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224"}
Nov 03 20:39:15.179796 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.179754    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:15.247184 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.247149    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.247361 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.247344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djxhv\" (UniqueName: \"kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.247456 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.247438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.247983 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.247960    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.250698 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.250669    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.298089 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.297982    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" podStartSLOduration=2.297969534 podStartE2EDuration="2.297969534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:15.296300983 +0000 UTC m=+42.017311227" watchObservedRunningTime="2026-11-03 20:39:15.297969534 +0000 UTC m=+42.018979819"
Nov 03 20:39:15.309077 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.309054    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djxhv\" (UniqueName: \"kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.324631 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.324608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:15.535495 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.535457    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:15.535673 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.535527    2578 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:15.535995 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.535979    2578 scope.go:122] "RemoveContainer" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"
Nov 03 20:39:15.536223 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:15.536200    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-xv2zb_openshift-console-operator(6d37aa0c-7893-407a-a52a-46d8a1ffd00e)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" podUID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e"
Nov 03 20:39:15.601548 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.601473    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:15.902484 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.902389    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:15.992991 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:15.992961    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-r2rmg"]
Nov 03 20:39:15.996094 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:15.996069    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e62edc4_aa68_49ad_ae64_909e88517f0f.slice/crio-6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838 WatchSource:0}: Error finding container 6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838: Status 404 returned error can't find the container with id 6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838
Nov 03 20:39:16.188855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.188777    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg" event={"ID":"9e62edc4-aa68-49ad-ae64-909e88517f0f","Type":"ContainerStarted","Data":"9d288f68650edfb20df13d9cabfc96421df616c21acee56386e28410fc3789f6"}
Nov 03 20:39:16.188992 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.188851    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg" event={"ID":"9e62edc4-aa68-49ad-ae64-909e88517f0f","Type":"ContainerStarted","Data":"6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838"}
Nov 03 20:39:16.305360 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.305312    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg" podStartSLOduration=2.305297975 podStartE2EDuration="2.305297975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:16.303642689 +0000 UTC m=+43.024652933" watchObservedRunningTime="2026-11-03 20:39:16.305297975 +0000 UTC m=+43.026308218"
Nov 03 20:39:16.646990 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.646960    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:16.814219 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.814185    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:16.963879 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.963802    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:16.963879 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:16.963861    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:16.964082 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:16.963956    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 03 20:39:16.964082 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:16.964014    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:20.96400153 +0000 UTC m=+47.685011754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : configmap "kube-rbac-proxy" not found
Nov 03 20:39:16.964082 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:16.963956    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:16.964082 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:16.964068    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:20.964051795 +0000 UTC m=+47.685062019 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:17.194428 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.194386    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f" event={"ID":"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe","Type":"ContainerStarted","Data":"8465da5aff86ba445e3b97ef05f0017d96990a2b708b005bcaeff975b904af0e"}
Nov 03 20:39:17.195968 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.195939    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w" event={"ID":"8e590e6f-ebec-4c39-8f00-d9460e3aeb57","Type":"ContainerStarted","Data":"773a3831e8090637ff062b83946b95520c104f699627888fc1a5b78dd37558ed"}
Nov 03 20:39:17.296931 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.296869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:17.314042 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.313998    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f" podStartSLOduration=2.4442362810000002 podStartE2EDuration="4.313983689s" totalImagesPullingTime="1.869747408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:17.312466635 +0000 UTC m=+44.033476877" watchObservedRunningTime="2026-11-03 20:39:17.313983689 +0000 UTC m=+44.034993934"
Nov 03 20:39:17.427242 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.427191    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w" podStartSLOduration=2.256435283 podStartE2EDuration="4.427177924s" totalImagesPullingTime="2.170742641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:17.426244192 +0000 UTC m=+44.147254434" watchObservedRunningTime="2026-11-03 20:39:17.427177924 +0000 UTC m=+44.148188166"
Nov 03 20:39:17.448867 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.448834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:17.752607 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.752505    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:17.907632 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:17.907603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:18.056374 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:18.056310    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:18.199674 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:18.199644    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:18.350678 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:18.350603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:18.774442 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:18.774419    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-66w89_84298777-7d2a-4d90-80aa-0cd0c65e04f3/dns-node-resolver/0.log"
Nov 03 20:39:19.300772 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:19.300740    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:19.489975 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:19.489947    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:19.691266 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:19.691195    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:19.774510 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:19.774484    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4cjfz_87fef4f1-d40d-4a12-9e0f-371838f9ecfa/node-ca/0.log"
Nov 03 20:39:19.889631 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:19.889601    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:20.086780 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.086744    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:20.261227 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.261201    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:20.405138 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.405064    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:20.574765 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.574740    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6sv74_91453c10-6f59-4fd5-bf2a-69c2c3944375/serve-healthcheck-canary/0.log"
Nov 03 20:39:20.775585 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.775547    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-9zp4f_78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe/migrator/0.log"
Nov 03 20:39:20.975082 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:20.975054    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-wjr2w_8e590e6f-ebec-4c39-8f00-d9460e3aeb57/migrator/0.log"
Nov 03 20:39:21.001194 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.001167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:21.001307 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.001197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:21.001307 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.001221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:21.001402 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.001303    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:21.001402 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.001372    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls podName:b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:29.001356078 +0000 UTC m=+55.722366333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tph4g" (UID: "b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97") : secret "insights-runtime-extractor-tls" not found
Nov 03 20:39:21.001701 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.001680    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:21.001760 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.001744    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:21.102173 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:21.102173 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102148    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102179    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102262    2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102324    2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102345    2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102345    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 03 20:39:21.102368 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102328    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert podName:b258928c-3513-426a-b50b-dd0f29f24a21 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:37.102313852 +0000 UTC m=+63.823324077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-4qvq7" (UID: "b258928c-3513-426a-b50b-dd0f29f24a21") : secret "networking-console-plugin-cert" not found
Nov 03 20:39:21.102725 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102390    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls podName:f76fe3d1-1a83-4e2b-867c-792eba90309f nodeName:}" failed. No retries permitted until 2026-11-03 20:39:37.102375636 +0000 UTC m=+63.823385860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-lbb9n" (UID: "f76fe3d1-1a83-4e2b-867c-792eba90309f") : secret "cluster-monitoring-operator-tls" not found
Nov 03 20:39:21.102725 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102412    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls podName:60bfa38c-92bc-4bef-ac05-18410621ea88 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:37.102396702 +0000 UTC m=+63.823406926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls") pod "dns-default-zjglm" (UID: "60bfa38c-92bc-4bef-ac05-18410621ea88") : secret "dns-default-metrics-tls" not found
Nov 03 20:39:21.102725 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:21.102725 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:21.102478    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls podName:657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d nodeName:}" failed. No retries permitted until 2026-11-03 20:39:37.102471894 +0000 UTC m=+63.823482114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-qmlbq" (UID: "657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d") : secret "samples-operator-tls" not found
Nov 03 20:39:21.102725 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.102598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:21.104789 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.104761    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:21.104990 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.104966    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:21.105084 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.105068    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:21.148332 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.148309    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:21.155466 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.155445    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:21.182053 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.182031    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx_bab0970d-6322-42d0-a66b-6d0acba24275/kube-storage-version-migrator-operator/0.log"
Nov 03 20:39:21.291014 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.290984    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:21.393992 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.393928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:21.488981 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.488946    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:21.660397 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.660376    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:39:21.663213 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:21.663185    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1747a6b1_0e38_4948_baa2_a75cbe265752.slice/crio-ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516 WatchSource:0}: Error finding container ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516: Status 404 returned error can't find the container with id ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516
Nov 03 20:39:21.690570 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.690538    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:21.710938 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.710668    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:39:21.712548 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:21.712526    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f528bfd_d20d_4fe1_81ae_754987ca453f.slice/crio-4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9 WatchSource:0}: Error finding container 4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9: Status 404 returned error can't find the container with id 4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9
Nov 03 20:39:21.789380 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.789353    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5769c487dc-z9pqs"]
Nov 03 20:39:21.792298 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:21.792275    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2480e6_eaa3_402a_b3fc_2c63f26ff172.slice/crio-268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b WatchSource:0}: Error finding container 268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b: Status 404 returned error can't find the container with id 268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b
Nov 03 20:39:21.889759 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:21.889523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:22.089224 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.089171    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:22.216235 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.216194    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" event={"ID":"0f528bfd-d20d-4fe1-81ae-754987ca453f","Type":"ContainerStarted","Data":"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"}
Nov 03 20:39:22.216405 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.216244    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" event={"ID":"0f528bfd-d20d-4fe1-81ae-754987ca453f","Type":"ContainerStarted","Data":"4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9"}
Nov 03 20:39:22.216405 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.216341    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:22.217967 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.217946    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" event={"ID":"1747a6b1-0e38-4948-baa2-a75cbe265752","Type":"ContainerStarted","Data":"4704c02d9eb88544e953bbff27017713c05ea0469a9b67a25f726d7be26a388c"}
Nov 03 20:39:22.218083 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.217977    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" event={"ID":"1747a6b1-0e38-4948-baa2-a75cbe265752","Type":"ContainerStarted","Data":"ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516"}
Nov 03 20:39:22.218083 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.218014    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:22.219422 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.219404    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5769c487dc-z9pqs" event={"ID":"4c2480e6-eaa3-402a-b3fc-2c63f26ff172","Type":"ContainerStarted","Data":"e9505720eb4ec5db31a5f313d83c789df6804a85fd4fd6f7e1dc848bc1086b0c"}
Nov 03 20:39:22.219490 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.219433    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5769c487dc-z9pqs" event={"ID":"4c2480e6-eaa3-402a-b3fc-2c63f26ff172","Type":"ContainerStarted","Data":"268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b"}
Nov 03 20:39:22.234677 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.234653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:22.338199 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.338145    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" podStartSLOduration=39.338129142 podStartE2EDuration="39.338129142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:22.336926935 +0000 UTC m=+49.057937192" watchObservedRunningTime="2026-11-03 20:39:22.338129142 +0000 UTC m=+49.059139385"
Nov 03 20:39:22.394790 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.394722    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:22.397344 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.397322    2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:22.458097 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.458045    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" podStartSLOduration=362.458033106 podStartE2EDuration="6m2.458033106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:22.457332115 +0000 UTC m=+49.178342373" watchObservedRunningTime="2026-11-03 20:39:22.458033106 +0000 UTC m=+49.179043350"
Nov 03 20:39:22.577218 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:22.577173    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/router-default-5769c487dc-z9pqs" podStartSLOduration=362.577161108 podStartE2EDuration="6m2.577161108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:22.576921238 +0000 UTC m=+49.297931493" watchObservedRunningTime="2026-11-03 20:39:22.577161108 +0000 UTC m=+49.298171351"
Nov 03 20:39:23.223808 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.223778    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:23.224841 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.224822    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:23.289368 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.289336    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:23.489790 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.489694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:23.689140 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.689109    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:23.823338 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.823260    2578 scope.go:122] "RemoveContainer" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"
Nov 03 20:39:23.836244 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.836172    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:23.977075 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:23.977043    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:24.119793 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.119720    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:24.229032 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229007    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:24.229430 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229412    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/1.log"
Nov 03 20:39:24.229470 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229443    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" containerID="3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895" exitCode=255
Nov 03 20:39:24.229541 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229520    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" event={"ID":"6d37aa0c-7893-407a-a52a-46d8a1ffd00e","Type":"ContainerDied","Data":"3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895"}
Nov 03 20:39:24.229617 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229594    2578 scope.go:122] "RemoveContainer" containerID="f9ee096074ddaf6741b4f49e88dbec9951fff482669b22cea2cb07e7f95ca478"
Nov 03 20:39:24.229923 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:24.229870    2578 scope.go:122] "RemoveContainer" containerID="3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895"
Nov 03 20:39:24.230089 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:24.230062    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-xv2zb_openshift-console-operator(6d37aa0c-7893-407a-a52a-46d8a1ffd00e)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" podUID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e"
Nov 03 20:39:25.234998 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.234972    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:25.296626 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.296598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:25.489338 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.489269    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:25.535314 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.535292    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:25.535432 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.535336    2578 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:25.535650 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.535635    2578 scope.go:122] "RemoveContainer" containerID="3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895"
Nov 03 20:39:25.535847 ip-10-0-141-248 kubenswrapper[2578]: E1103 20:39:25.535830    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-xv2zb_openshift-console-operator(6d37aa0c-7893-407a-a52a-46d8a1ffd00e)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" podUID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e"
Nov 03 20:39:25.689418 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.689391    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:25.889809 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:25.889729    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:26.091957 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:26.091926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:26.832101 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:26.832065    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:27.288692 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:27.288654    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:27.490452 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:27.490423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:27.688122 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:27.688057    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:27.888900 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:27.888865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:28.029716 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:28.029688    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:28.171468 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:28.171423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:29.067478 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:29.067446    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:29.069855 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:29.069830    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:29.173628 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:29.173600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:29.553603 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:29.553577    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tph4g"]
Nov 03 20:39:29.556102 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:29.556074    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ce0b31_7cb3_47f5_82a5_7ea11cfc5a97.slice/crio-0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c WatchSource:0}: Error finding container 0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c: Status 404 returned error can't find the container with id 0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c
Nov 03 20:39:30.078729 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:30.078703    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:30.253804 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:30.253770    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tph4g" event={"ID":"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97","Type":"ContainerStarted","Data":"71c1cbead764e75233e96d049de3459320c484757ae9d7793d37b2f5375b0131"}
Nov 03 20:39:30.253950 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:30.253820    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tph4g" event={"ID":"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97","Type":"ContainerStarted","Data":"0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c"}
Nov 03 20:39:31.259100 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:31.259074    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tph4g" event={"ID":"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97","Type":"ContainerStarted","Data":"6a89f064f3a5ced9086d0424d683ea9dd40b5ad75532ed4e41db7544ccca9f38"}
Nov 03 20:39:33.269159 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:33.269122    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tph4g" event={"ID":"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97","Type":"ContainerStarted","Data":"f0496708a16aa32a181cd9a243552277df7513b418c9e2c196d55a583c56f65a"}
Nov 03 20:39:33.387327 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:33.387265    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tph4g" podStartSLOduration=17.532778416 podStartE2EDuration="20.38725004s" totalImagesPullingTime="2.854471624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:39:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:33.386471759 +0000 UTC m=+60.107482012" watchObservedRunningTime="2026-11-03 20:39:33.38725004 +0000 UTC m=+60.108260283"
Nov 03 20:39:37.135077 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.135045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:37.135512 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.135084    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:37.135512 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.135123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:37.135512 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.135150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:37.137575 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.137526    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:37.137675 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.137605    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:37.138031 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.138012    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:37.138087 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.138015    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:37.304488 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.304464    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:37.315512 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.315489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:37.342734 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.342497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:37.425380 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.425359    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9ll5z\""
Nov 03 20:39:37.434622 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.434599    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:37.726251 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.726218    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"]
Nov 03 20:39:37.766462 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:37.766438    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76fe3d1_1a83_4e2b_867c_792eba90309f.slice/crio-a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940 WatchSource:0}: Error finding container a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940: Status 404 returned error can't find the container with id a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940
Nov 03 20:39:37.775274 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.775215    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"]
Nov 03 20:39:37.776802 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.776779    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"]
Nov 03 20:39:37.777262 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:37.777242    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb258928c_3513_426a_b50b_dd0f29f24a21.slice/crio-b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39 WatchSource:0}: Error finding container b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39: Status 404 returned error can't find the container with id b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39
Nov 03 20:39:37.837302 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:37.837281    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zjglm"]
Nov 03 20:39:37.839604 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:37.839583    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bfa38c_92bc_4bef_ac05_18410621ea88.slice/crio-79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038 WatchSource:0}: Error finding container 79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038: Status 404 returned error can't find the container with id 79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038
Nov 03 20:39:38.287339 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.287303    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7" event={"ID":"b258928c-3513-426a-b50b-dd0f29f24a21","Type":"ContainerStarted","Data":"b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39"}
Nov 03 20:39:38.288462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.288429    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq" event={"ID":"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d","Type":"ContainerStarted","Data":"a74f90854ca418b10f3348c2898022bb0aec55e888957f22e54e59d83757ea17"}
Nov 03 20:39:38.289627 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.289598    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n" event={"ID":"f76fe3d1-1a83-4e2b-867c-792eba90309f","Type":"ContainerStarted","Data":"a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940"}
Nov 03 20:39:38.290691 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.290673    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjglm" event={"ID":"60bfa38c-92bc-4bef-ac05-18410621ea88","Type":"ContainerStarted","Data":"79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038"}
Nov 03 20:39:38.748781 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.748337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:38.748781 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.748394    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:38.775033 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.775000    2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:39:38.781552 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.781520    2578 patch_prober.go:29] interesting pod/image-registry-5cf6c99f88-pj2bd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 20:39:38.781716 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.781590    2578 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:39:38.809101 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.809074    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:38.814305 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.814239    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:38.823245 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.823163    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:38.827978 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.827921    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:38.838079 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.838031    2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:38.992882 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:38.992808    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:39.000402 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.000377    2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:39.002623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.002601    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:39.010208 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.010006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:39.044918 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.044888    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:39.195027 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.194935    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:39.398623 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.398533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:39.465499 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.465465    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ffxrp"]
Nov 03 20:39:39.480515 ip-10-0-141-248 kubenswrapper[2578]: W1103 20:39:39.480449    2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb768b17b_8769_4002_8c89_3bee5fda2060.slice/crio-d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791 WatchSource:0}: Error finding container d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791: Status 404 returned error can't find the container with id d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791
Nov 03 20:39:39.513648 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.513574    2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xjbt9"]
Nov 03 20:39:39.544910 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:39.544886    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:40.302594 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:40.302294    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7" event={"ID":"b258928c-3513-426a-b50b-dd0f29f24a21","Type":"ContainerStarted","Data":"3912cccf3811506a6b3707761e5e3b9a263144281db32f752a0d9635e77cd784"}
Nov 03 20:39:40.306519 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:40.306467    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjbt9" event={"ID":"5713f5a0-c8bf-44c1-aeff-c9c580746c45","Type":"ContainerStarted","Data":"e7babdeb287a7093e13d1d2e44e77ebfbc0060d6969ab40a5b861b1451339613"}
Nov 03 20:39:40.308374 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:40.308285    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffxrp" event={"ID":"b768b17b-8769-4002-8c89-3bee5fda2060","Type":"ContainerStarted","Data":"d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791"}
Nov 03 20:39:41.161415 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.161350    2578 patch_prober.go:29] interesting pod/image-registry-98474cc5c-vc78r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 20:39:41.161415 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.161402    2578 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:39:41.320329 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.319643    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq" event={"ID":"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d","Type":"ContainerStarted","Data":"f4b1f40e4d2c52e0320dfb16a802a64aee878c2679444459d8e715d7010b6fc4"}
Nov 03 20:39:41.320329 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.319729    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq" event={"ID":"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d","Type":"ContainerStarted","Data":"774dcb5ce524069f6c786e5e50a5c0ca651f10edd8cfdf4e2f20f2904e1dfb48"}
Nov 03 20:39:41.323938 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.323513    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n" event={"ID":"f76fe3d1-1a83-4e2b-867c-792eba90309f","Type":"ContainerStarted","Data":"b3d5f5a152f0dac4b4e1f7ff2dc421bd58a247676e679a9c76c8caa79d6b8b0a"}
Nov 03 20:39:41.326396 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.326328    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffxrp" event={"ID":"b768b17b-8769-4002-8c89-3bee5fda2060","Type":"ContainerStarted","Data":"75322cb23ed8dc17fe937c133723cd13bb46c9e3a8bffbc299a97f8834a123aa"}
Nov 03 20:39:41.329075 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.328964    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjglm" event={"ID":"60bfa38c-92bc-4bef-ac05-18410621ea88","Type":"ContainerStarted","Data":"f147bec7e3fa76269ea60b7e5bd9256ea091bf39da5ffbc7d31770956c8b940d"}
Nov 03 20:39:41.329075 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.328995    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zjglm" event={"ID":"60bfa38c-92bc-4bef-ac05-18410621ea88","Type":"ContainerStarted","Data":"6082a23b4ac4ecc3f0be5bf52e8772dee2ab61d6709c134da56d0e5c3d6e6bc5"}
Nov 03 20:39:41.329075 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.329020    2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:41.472654 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.472595    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7" podStartSLOduration=343.510098291 podStartE2EDuration="5m45.472539418s" totalImagesPullingTime="1.962441127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:40.426714039 +0000 UTC m=+67.147724283" watchObservedRunningTime="2026-11-03 20:39:41.472539418 +0000 UTC m=+68.193549662"
Nov 03 20:39:41.474827 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.474610    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq" podStartSLOduration=378.822000664 podStartE2EDuration="6m21.474593116s" totalImagesPullingTime="2.652592452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:41.472207335 +0000 UTC m=+68.193217582" watchObservedRunningTime="2026-11-03 20:39:41.474593116 +0000 UTC m=+68.195603362"
Nov 03 20:39:41.589506 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.589447    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n" podStartSLOduration=378.840031936 podStartE2EDuration="6m21.58943201s" totalImagesPullingTime="2.749400074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:41.58915908 +0000 UTC m=+68.310169319" watchObservedRunningTime="2026-11-03 20:39:41.58943201 +0000 UTC m=+68.310442257"
Nov 03 20:39:41.707542 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.707486    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-zjglm" podStartSLOduration=33.875707064 podStartE2EDuration="36.707471944s" totalImagesPullingTime="2.83176488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:41.70675376 +0000 UTC m=+68.427764005" watchObservedRunningTime="2026-11-03 20:39:41.707471944 +0000 UTC m=+68.428482186"
Nov 03 20:39:41.764634 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.764516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:41.911864 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:41.911831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:42.057356 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:42.057280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:42.335317 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:42.335208    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ffxrp" event={"ID":"b768b17b-8769-4002-8c89-3bee5fda2060","Type":"ContainerStarted","Data":"29ac8656ec8ff2346e74f7ee8f031c21581ad9d3dd92edb07885775faf8ec5f6"}
Nov 03 20:39:42.457213 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:42.457164    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ffxrp" podStartSLOduration=65.773651656 podStartE2EDuration="1m7.457148154s" totalImagesPullingTime="1.683496498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:42.454628658 +0000 UTC m=+69.175638904" watchObservedRunningTime="2026-11-03 20:39:42.457148154 +0000 UTC m=+69.178158396"
Nov 03 20:39:42.839729 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:42.839678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:43.144763 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:43.144683    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:43.228630 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:43.228588    2578 patch_prober.go:29] interesting pod/image-registry-98474cc5c-vc78r container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 03 20:39:43.228808 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:43.228648    2578 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:39:43.764412 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:43.764372    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:45.350462 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:45.350420    2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xjbt9" event={"ID":"5713f5a0-c8bf-44c1-aeff-c9c580746c45","Type":"ContainerStarted","Data":"c8b3fdee509989503865369cc2cc7e8750ae6bb800831a87d5c6559862d10587"}
Nov 03 20:39:45.470465 ip-10-0-141-248 kubenswrapper[2578]: I1103 20:39:45.470412    2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xjbt9" podStartSLOduration=64.799816573 podStartE2EDuration="1m10.470399177s" totalImagesPullingTime="5.670582604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:45.469101603 +0000 UTC m=+72.190111846" watchObservedRunningTime="2026-11-03 20:39:45.470399177 +0000 UTC m=+72.191409419"
Nov 03 20:39:45.494703 ip-10-0-141-248 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 20:39:45.505930 ip-10-0-141-248 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 20:39:45.506100 ip-10-0-141-248 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 20:39:45.506159 ip-10-0-141-248 systemd[1]: kubelet.service: Consumed 4.625s CPU time.
Nov 03 20:39:45.516261 ip-10-0-141-248 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 20:39:45.592542 ip-10-0-141-248 kubenswrapper[5915]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:45.592542 ip-10-0-141-248 kubenswrapper[5915]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 20:39:45.592542 ip-10-0-141-248 kubenswrapper[5915]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:45.592542 ip-10-0-141-248 kubenswrapper[5915]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594473    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594567    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594572    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594575    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594578    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594581    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594584    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594586    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594589    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594592    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.594552 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594595    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594598    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594601    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594603    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594606    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594609    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594612    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594614    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594617    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594620    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594622    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594625    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594627    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594630    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594632    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594634    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594641    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594643    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594646    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594649    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.594886 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594652    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594655    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594657    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594660    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594663    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594665    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594668    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594671    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594673    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594676    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594678    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594681    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594684    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594686    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594690    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594692    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594695    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594697    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594700    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594703    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.595389 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594705    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594707    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594710    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594713    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594716    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594718    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594721    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594723    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594725    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594728    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594730    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594733    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594736    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594741    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594745    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594748    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594751    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594754    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594757    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.595921 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594760    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594763    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594767    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594771    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594773    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594776    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594779    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594781    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594784    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594787    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594790    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594792    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594795    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594797    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594800    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594805    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594808    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594810    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594813    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594816    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.596402 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594818    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594821    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594824    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594826    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594829    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594832    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594835    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594837    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594840    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594843    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594845    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594848    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594851    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594853    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594856    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594858    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.594862    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595617    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595625    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595627    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.596912 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595630    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595640    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595644    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595646    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595649    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595651    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595654    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595660    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595662    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595665    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595667    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595670    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595672    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595675    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595677    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595680    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595683    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595688    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595692    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.597420 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595695    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595698    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595701    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595703    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595706    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595708    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595711    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595713    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595716    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595718    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595721    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595723    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595725    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595728    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595730    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595738    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595742    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595744    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595747    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.597917 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595749    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595754    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595758    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595761    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595764    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595767    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595770    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595772    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595775    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595778    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595780    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595783    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595785    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595788    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595790    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595793    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595795    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595797    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595801    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595803    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.598396 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595805    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595808    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595810    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595813    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595816    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595818    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595821    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595823    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595826    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595829    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595831    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595833    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595836    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595840    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595843    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595846    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595848    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595851    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595853    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595856    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.598941 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595858    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595861    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595864    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595866    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595869    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595871    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595874    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595876    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595879    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595881    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595884    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595886    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595889    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595891    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595893    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595896    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595898    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595901    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595903    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.599483 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595906    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595910    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595912    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595915    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595917    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.595920    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596461    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596468    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596471    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596486    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596490    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596493    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596496    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596499    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596502    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596504    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596507    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596509    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596512    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596515    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.599974 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596517    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596520    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596523    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596525    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596528    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596530    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596533    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596535    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596537    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596540    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596542    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596545    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596547    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596550    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596552    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596575    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596578    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596581    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596584    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596587    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.600488 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596589    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596592    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596595    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596597    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596600    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596602    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596604    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596607    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596609    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596612    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596614    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596616    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596619    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596621    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596623    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596626    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596628    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596631    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596633    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596636    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.601014 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596639    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596641    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596644    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596647    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596650    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596652    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596654    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596659    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596667    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596669    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596674    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596677    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596681    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596684    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596687    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596689    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596692    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596694    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.601828 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596697    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596699    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596702    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596704    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596707    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596710    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596712    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596715    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596718    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596720    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596723    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596725    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596728    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596730    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596733    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596735    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596738    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596740    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596743    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596745    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.602368 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596748    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596751    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596759    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596768    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596771    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596773    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596776    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596778    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596781    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596783    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596786    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596789    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596792    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.596794    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596905    5915 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596925    5915 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596932    5915 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596937    5915 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596941    5915 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596944    5915 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596948    5915 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 20:39:45.602898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596951    5915 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596954    5915 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596957    5915 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596961    5915 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596964    5915 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596967    5915 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596970    5915 flags.go:64] FLAG: --cgroup-root=""
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596972    5915 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596975    5915 flags.go:64] FLAG: --client-ca-file=""
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596978    5915 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596981    5915 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596988    5915 flags.go:64] FLAG: --cluster-domain=""
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596992    5915 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596995    5915 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.596998    5915 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597011    5915 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597016    5915 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597019    5915 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597021    5915 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597025    5915 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597028    5915 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597030    5915 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597033    5915 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597036    5915 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 20:39:45.603430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597039    5915 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597043    5915 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597046    5915 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597049    5915 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597052    5915 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597055    5915 flags.go:64] FLAG: --enable-server="true"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597058    5915 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597064    5915 flags.go:64] FLAG: --event-burst="100"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597067    5915 flags.go:64] FLAG: --event-qps="50"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597070    5915 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597073    5915 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597076    5915 flags.go:64] FLAG: --eviction-hard=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597080    5915 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597083    5915 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597086    5915 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597089    5915 flags.go:64] FLAG: --eviction-soft=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597091    5915 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597094    5915 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597097    5915 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597100    5915 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597104    5915 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597107    5915 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597109    5915 flags.go:64] FLAG: --feature-gates=""
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597114    5915 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597123    5915 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 20:39:45.604048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597126    5915 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597130    5915 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597133    5915 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597136    5915 flags.go:64] FLAG: --help="false"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597139    5915 flags.go:64] FLAG: --hostname-override="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597142    5915 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597145    5915 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597148    5915 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597151    5915 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597155    5915 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597158    5915 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597160    5915 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597163    5915 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597166    5915 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597169    5915 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597172    5915 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597175    5915 flags.go:64] FLAG: --kube-reserved=""
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597178    5915 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597180    5915 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597183    5915 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597186    5915 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597188    5915 flags.go:64] FLAG: --lock-file=""
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597191    5915 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597194    5915 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 20:39:45.604681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597197    5915 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597206    5915 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597209    5915 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597213    5915 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597216    5915 flags.go:64] FLAG: --logging-format="text"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597219    5915 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597223    5915 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597226    5915 flags.go:64] FLAG: --manifest-url=""
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597234    5915 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597239    5915 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597242    5915 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597246    5915 flags.go:64] FLAG: --max-pods="110"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597248    5915 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597251    5915 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597254    5915 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597257    5915 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597260    5915 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597263    5915 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597266    5915 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597272    5915 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597275    5915 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597278    5915 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597281    5915 flags.go:64] FLAG: --pod-cidr=""
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597284    5915 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597287    5915 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 20:39:45.605364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597289    5915 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597292    5915 flags.go:64] FLAG: --port="10250"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597295    5915 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597298    5915 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0922fba4f01df7461"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597301    5915 flags.go:64] FLAG: --qos-reserved=""
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597304    5915 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597307    5915 flags.go:64] FLAG: --register-node="true"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597310    5915 flags.go:64] FLAG: --register-with-taints=""
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597313    5915 flags.go:64] FLAG: --registry-burst="10"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597316    5915 flags.go:64] FLAG: --registry-qps="5"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597321    5915 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597323    5915 flags.go:64] FLAG: --reserved-memory=""
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597327    5915 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597332    5915 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597335    5915 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597338    5915 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597346    5915 flags.go:64] FLAG: --runonce="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597350    5915 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597352    5915 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597355    5915 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597358    5915 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597361    5915 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597364    5915 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597367    5915 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597370    5915 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597372    5915 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 20:39:45.606010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597375    5915 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597378    5915 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597380    5915 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597383    5915 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597392    5915 flags.go:64] FLAG: --system-cgroups=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597395    5915 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597400    5915 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597403    5915 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597405    5915 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597412    5915 flags.go:64] FLAG: --tls-min-version=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597415    5915 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597418    5915 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597420    5915 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597423    5915 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597426    5915 flags.go:64] FLAG: --v="2"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597431    5915 flags.go:64] FLAG: --version="false"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597436    5915 flags.go:64] FLAG: --vmodule=""
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597439    5915 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597443    5915 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597655    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597659    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597662    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597665    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597673    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.606673 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597676    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597679    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597681    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597685    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597689    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597692    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597695    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597698    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597701    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597703    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597706    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597709    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597711    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597714    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597717    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597719    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597723    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597727    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597730    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.607251 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597733    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597736    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597739    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597741    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597744    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597747    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597749    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597752    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597755    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597758    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597761    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597763    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597766    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597774    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597777    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597780    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597782    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597785    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597787    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597789    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.607768 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597792    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597795    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597797    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597800    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597802    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597805    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597808    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597810    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597813    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597815    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597817    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597820    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597823    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597825    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597828    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597830    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597833    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597835    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597837    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597840    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.608461 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597844    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597846    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597849    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597851    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597854    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597856    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597864    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597867    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597870    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597872    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597875    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597877    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597880    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597882    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597884    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597887    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597890    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597892    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597895    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.609159 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597897    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597900    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597902    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597904    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597907    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597909    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597912    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597914    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597917    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597919    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597922    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597925    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597927    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597931    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597934    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597937    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597939    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597942    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597944    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597946    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.609660 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597955    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597957    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.597960    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.597985    5915 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598028    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598032    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598036    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598041    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598043    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598046    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598049    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598051    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598054    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598056    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598058    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598061    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.610166 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598063    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598066    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598069    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598071    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598074    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598076    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598078    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598081    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598083    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598086    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598091    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598094    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598097    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598099    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598102    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598104    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598107    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598120    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598132    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.610689 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598135    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598137    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598140    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598143    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598146    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598149    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598151    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598154    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598157    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598159    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598162    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598164    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598167    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598169    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598172    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598174    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598177    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598179    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598182    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.611285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598184    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598186    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598189    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598191    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598196    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598199    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598201    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598203    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598206    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598208    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598211    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598213    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598222    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598225    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598227    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598230    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598232    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598236    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598239    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598241    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.611820 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598244    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598246    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598248    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598251    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598253    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598256    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598258    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598260    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598263    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598265    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598268    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598270    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598272    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598275    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598277    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598280    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598283    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598286    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598288    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.612354 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598291    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598295    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598298    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598301    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598304    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598306    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598314    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598317    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598320    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598322    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598327    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598329    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598332    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598334    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598338    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598342    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.598345    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.603093    5915 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.603109    5915 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 20:39:45.612891 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.603127    5915 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.603132    5915 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603196    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603201    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603204    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603207    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603209    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603212    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603215    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603219    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603223    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603226    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603229    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603232    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603234    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603237    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603240    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603242    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603245    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.613373 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603248    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603250    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603265    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603267    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603270    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603273    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603276    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603278    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603281    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603283    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603286    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603293    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603296    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603300    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603304    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603307    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603310    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603312    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603314    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.613896 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603317    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603319    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603322    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603324    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603327    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603329    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603332    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603334    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603337    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603339    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603342    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603345    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603347    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603350    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603352    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603354    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603357    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603360    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603362    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603365    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.614407 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603367    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603369    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603372    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603374    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603377    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603379    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603381    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603384    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603386    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603389    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603392    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603394    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603397    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603399    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603402    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603404    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603406    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603409    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603411    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.614943 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603414    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603416    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603418    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603421    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603424    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603426    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603429    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603431    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603434    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603437    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603439    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603442    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603444    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603447    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603450    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603452    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603455    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603457    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603460    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603463    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.615446 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603466    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603468    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603471    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603473    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603476    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603478    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603481    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603483    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603486    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603489    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603491    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.603518    5915 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603707    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603712    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603715    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603718    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.617193 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603721    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603725    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603727    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603730    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603732    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603735    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603738    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603740    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603742    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603745    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603747    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603750    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603752    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603755    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603757    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603760    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603762    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603764    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603767    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603769    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.617756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603772    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603774    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603777    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603779    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603782    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603784    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603787    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603789    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603792    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603797    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603801    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603805    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603807    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603810    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603813    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603815    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603818    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603821    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603823    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.618326 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603826    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603828    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603831    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603833    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603836    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603838    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603840    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603843    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603845    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603847    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603850    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603852    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603855    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603858    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603860    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603864    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603868    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603870    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603873    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.618906 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603876    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603878    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603880    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603884    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603887    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603889    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603892    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603895    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603897    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603900    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603903    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603905    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603907    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603910    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603913    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603916    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603918    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603921    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603923    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.619536 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603926    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603928    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603931    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603933    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603936    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603938    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603940    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603943    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603945    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603948    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603950    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603953    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603955    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603958    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603960    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603963    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603967    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603969    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603972    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603974    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.620168 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603977    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603979    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603982    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603985    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.603988    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.604011    5915 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604042    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604047    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604051    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604054    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604057    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604060    5915 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604062    5915 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604065    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604067    5915 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604070    5915 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:45.620844 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604072    5915 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604075    5915 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604077    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604080    5915 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604083    5915 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604085    5915 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604088    5915 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604090    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604092    5915 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604095    5915 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604097    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604100    5915 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604103    5915 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604105    5915 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604108    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604110    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604113    5915 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604115    5915 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604118    5915 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604120    5915 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:45.621353 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604123    5915 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604125    5915 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604128    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604130    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604134    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604136    5915 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604139    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604142    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604144    5915 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604147    5915 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604149    5915 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604151    5915 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604154    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604157    5915 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604159    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604161    5915 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604165    5915 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604168    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604171    5915 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:45.622029 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604174    5915 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604177    5915 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604179    5915 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604182    5915 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604184    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604188    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604190    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604192    5915 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604195    5915 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604198    5915 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604200    5915 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604203    5915 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604205    5915 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604207    5915 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604210    5915 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604213    5915 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604215    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604219    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604221    5915 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:45.622671 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604224    5915 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604226    5915 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604228    5915 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604232    5915 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604236    5915 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604238    5915 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604241    5915 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604244    5915 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604246    5915 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604249    5915 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604252    5915 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604254    5915 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604257    5915 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604260    5915 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604263    5915 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604265    5915 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604268    5915 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604270    5915 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604273    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:45.623285 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604275    5915 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604278    5915 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604280    5915 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604282    5915 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604285    5915 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604287    5915 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604289    5915 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604292    5915 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604294    5915 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604297    5915 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604301    5915 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604305    5915 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604307    5915 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604310    5915 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604312    5915 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604314    5915 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604317    5915 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604319    5915 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:45.604322    5915 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.604395    5915 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 20:39:45.623774 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605346    5915 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605407    5915 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605604    5915 server.go:1014] "Starting client certificate rotation"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605755    5915 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605804    5915 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 20:33:34 +0000 UTC" deadline="2028-04-04 16:41:36.44716149 +0000 UTC"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.605860    5915 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12428h1m50.84130282s"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.606118    5915 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.607200    5915 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.608406    5915 log.go:25] "Validated CRI v1 runtime API"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.610314    5915 log.go:25] "Validated CRI v1 image API"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.611037    5915 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 20:39:45.624258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.613418    5915 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7a5ca57b-1524-4e95-bfa5-03384ba88f98:/dev/nvme0n1p3 c5baa399-1031-4f77-b14d-ca276d36e0fe:/dev/nvme0n1p4]
Nov 03 20:39:45.624623 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.613438    5915 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c/userdata/shm major:0 minor:413 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1635c73de0734221b3fdd22ba9a30731b33a8d1967bce0f0ee7675c47a951cc9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1635c73de0734221b3fdd22ba9a30731b33a8d1967bce0f0ee7675c47a951cc9/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b/userdata/shm major:0 minor:386 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f/userdata/shm major:0 minor:223 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8/userdata/shm major:0 minor:345 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9/userdata/shm major:0 minor:384 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5c92695d378a43c8d841ce5f8f23e24c7e07287b695ec1a9aa2c19b3019e3a45/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5c92695d378a43c8d841ce5f8f23e24c7e07287b695ec1a9aa2c19b3019e3a45/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838/userdata/shm major:0 minor:357 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038/userdata/shm major:0 minor:443 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a/userdata/shm major:0 minor:338 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940/userdata/shm major:0 minor:439 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a74f90854ca418b10f3348c2898022bb0aec55e888957f22e54e59d83757ea17/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a74f90854ca418b10f3348c2898022bb0aec55e888957f22e54e59d83757ea17/userdata/shm major:0 minor:437 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516/userdata/shm major:0 minor:382 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39/userdata/shm major:0 minor:441 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791/userdata/shm major:0 minor:460 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e7babdeb287a7093e13d1d2e44e77ebfbc0060d6969ab40a5b861b1451339613/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e7babdeb287a7093e13d1d2e44e77ebfbc0060d6969ab40a5b861b1451339613/userdata/shm major:0 minor:459 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/kube-api-access-zwn5d:{mountpoint:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/kube-api-access-zwn5d major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/registry-tls major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/kube-api-access-gbz78:{mountpoint:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/kube-api-access-gbz78 major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/registry-tls major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~projected/kube-api-access-hxkq8:{mountpoint:/var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~projected/kube-api-access-hxkq8 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb/volumes/kubernetes.io~projected/kube-api-access-vlhzq:{mountpoint:/var/lib/kubelet/pods/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb/volumes/kubernetes.io~projected/kube-api-access-vlhzq major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30b966c5-d3d0-4743-a041-fe5f0f196fd1/volumes/kubernetes.io~projected/kube-api-access-72v9b:{mountpoint:/var/lib/kubelet/pods/30b966c5-d3d0-4743-a041-fe5f0f196fd1/volumes/kubernetes.io~projected/kube-api-access-72v9b major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/476f59f4-3347-4e66-b0ea-51add44cfc27/volumes/kubernetes.io~projected/kube-api-access-f46q7:{mountpoint:/var/lib/kubelet/pods/476f59f4-3347-4e66-b0ea-51add44cfc27/volumes/kubernetes.io~projected/kube-api-access-f46q7 major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~projected/kube-api-access-4cr8r:{mountpoint:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~projected/kube-api-access-4cr8r major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/default-certificate major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/metrics-certs major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/stats-auth major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~projected/kube-api-access-r96ml:{mountpoint:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~projected/kube-api-access-r96ml major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51834fed-df6e-444b-9c2d-4501e262df12/volumes/kubernetes.io~projected/kube-api-access-lcwcr:{mountpoint:/var/lib/kubelet/pods/51834fed-df6e-444b-9c2d-4501e262df12/volumes/kubernetes.io~projected/kube-api-access-lcwcr major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54725514-9087-42f7-8c47-5a995d71a0ee/volumes/kubernetes.io~projected/kube-api-access-8cqjr:{mountpoint:/var/lib/kubelet/pods/54725514-9087-42f7-8c47-5a995d71a0ee/volumes/kubernetes.io~projected/kube-api-access-8cqjr major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5713f5a0-c8bf-44c1-aeff-c9c580746c45/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/5713f5a0-c8bf-44c1-aeff-c9c580746c45/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:458 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~projected/kube-api-access-v9cbl:{mountpoint:/var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~projected/kube-api-access-v9cbl major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~secret/metrics-tls major:0 minor:434 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~projected/kube-api-access-gf9s5:{mountpoint:/var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~projected/kube-api-access-gf9s5 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~secret/samples-operator-tls:{mountpoint:/var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~secret/samples-operator-tls major:0 minor:430 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66a0afd2-dcb3-48ab-905d-4e4abab2fe97/volumes/kubernetes.io~projected/kube-api-access-hwvc7:{mountpoint:/var/lib/kubelet/pods/66a0afd2-dcb3-48ab-905d-4e4abab2fe97/volumes/kubernetes.io~projected/kube-api-access-hwvc7 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c893222-ab83-4a89-98e2-169613906e67/volumes/kubernetes.io~projected/kube-api-access-xvrs8:{mountpoint:/var/lib/kubelet/pods/6c893222-ab83-4a89-98e2-169613906e67/volumes/kubernetes.io~projected/kube-api-access-xvrs8 major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~projected/kube-api-access-s8qts:{mountpoint:/var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~projected/kube-api-access-s8qts major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~secret/serving-cert major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe/volumes/kubernetes.io~projected/kube-api-access-ssmxp:{mountpoint:/var/lib/kubelet/pods/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe/volumes/kubernetes.io~projected/kube-api-access-ssmxp major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~projected/kube-api-access-76xpb:{mountpoint:/var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~projected/kube-api-access-76xpb major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/84298777-7d2a-4d90-80aa-0cd0c65e04f3/volumes/kubernetes.io~projected/kube-api-access-2kx9d:{mountpoint:/var/lib/kubelet/pods/84298777-7d2a-4d90-80aa-0cd0c65e04f3/volumes/kubernetes.io~projected/kube-api-access-2kx9d major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87fef4f1-d40d-4a12-9e0f-371838f9ecfa/volumes/kubernetes.io~projected/kube-api-access-bbjdn:{mountpoint:/var/lib/kubelet/pods/87fef4f1-d40d-4a12-9e0f-371838f9ecfa/volumes/kubernetes.io~projected/kube-api-access-bbjdn major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8e590e6f-ebec-4c39-8f00-d9460e3aeb57/volumes/kubernetes.io~projected/kube-api-access-kj8wv:{mountpoint:/var/lib/kubelet/pods/8e590e6f-ebec-4c39-8f00-d9460e3aeb57/volumes/kubernetes.io~projected/kube-api-access-kj8wv major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90155d8a-eff2-4b22-ac1b-975bffe1d644/volumes/kubernetes.io~projected/kube-api-access-47h9w:{mountpoint:/var/lib/kubelet/pods/90155d8a-eff2-4b22-ac1b-975bffe1d644/volumes/kubernetes.io~projected/kube-api-access-47h9w major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~projected/kube-api-access-2ggk7:{mountpoint:/var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~projected/kube-api-access-2ggk7 major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~secret/cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32/volumes/kubernetes.io~projected/kube-api-access-jhxxb:{mountpoint:/var/lib/kubelet/pods/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32/volumes/kubernetes.io~projected/kube-api-access-jhxxb major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~projected/kube-api-access-djxhv:{mountpoint:/var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~projected/kube-api-access-djxhv major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~secret/signing-key major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b258928c-3513-426a-b50b-dd0f29f24a21/volumes/kubernetes.io~secret/networking-console-plugin-cert:{mountpoint:/var/lib/kubelet/pods/b258928c-3513-426a-b50b-dd0f29f24a21/volumes/kubernetes.io~secret/networking-console-plugin-cert major:0 minor:435 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0/volumes/kubernetes.io~projected/kube-api-access-zwqr7:{mountpoint:/var/lib/kubelet/pods/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0/volumes/kubernetes.io~projected/kube-api-access-zwqr7 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~projected/kube-api-access-979r2:{mountpoint:/var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~projected/kube-api-access-979r2 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~secret/metrics-certs major:0 minor:457 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~projected/kube-api-access-kp5qz:{mountpoint:/var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~projected/kube-api-access-kp5qz major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:408 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~projected/kube-api-access-wrl4g:{mountpoint:/var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~projected/kube-api-access-wrl4g major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~secret/serving-cert major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/tmp major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~projected/kube-api-access-6fns9:{mountpoint:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~projected/kube-api-access-6fns9 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6d5af76-ea42-41d7-9920-db291e351c53/volumes/kubernetes.io~projected/kube-api-access-s4j8h:{mountpoint:/var/lib/kubelet/pods/c6d5af76-ea42-41d7-9920-db291e351c53/volumes/kubernetes.io~projected/kube-api-access-s4j8h major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~projected/kube-api-access-mwzrq:{mountpoint:/var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~projected/kube-api-access-mwzrq major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls:{mountpoint:/var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls major:0 minor:436 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ff27c4bf-c1d8-47f8-9737-24510e201615/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ff27c4bf-c1d8-47f8-9737-24510e201615/volumes/kubernetes.io~secret/agent-certs major:0 minor:49 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/8c7c44046f7e8dcd8b8af1680dc16bd3f233eead41f9c47126168d30b889627b/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/6a4ba587a23456faefe1f27040646742eb740d86fccd3c461480c0c4ffb659c2/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/38ad84030db35f4c4abeab988e85217dc0c10c682fb7f3152d57a05827e21ccc/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/containers/storage/overlay/9d076c86aa1f7503714ea2754c060a9c2eb836a6fe12bdd8726d3b8564b8e0e8/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/5ee8345a2607b8368782e76441a71f4d4852bb3c6346d0f8f636dc6f8f218e8a/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/1d048141f03ed93e43d65a53ec5e8860c43d83382f8a5e266e704e229f156060/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-122:{mountpoint:/var/lib/containers/storage/overlay/5c860fa821c5242fa59a6a6209b31b29c50bf7f54e0ea6e671ed148abbef3017/merged major:0 minor:122 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/58477c4cd63d472f89eff9ba6538b207e9b8a0f83720d6c9f9c050b39ff88915/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/cd525ee6975eac0c08fed6dc200ea07347be549bfd8da007cad04855fed4a2c8/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/c2315d6d21f2957b44171e422dacab63a28b7596b9a8fc1e3cc9e91c2e14844a/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/dce23c2b597ad18e03da43a3e527782e952ca7d36c161cfa5cd860e9968e9068/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/9f0c504cfdd9e8f48b40e69b3badca97349a697fe6a1742d28b207ac88fec808/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/45b34b599ccbda3c4ec66e98d19ef63a822f937f552a279f845a70aeaca324bb/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/0b101c11122f9a38e4f0a6481471b10441fcb57ef7eca10e59c33cb9dc249666/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/9ec53f017e364c6565adb950260f4acb753e3ba2887d777e348c062e36879cf2/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/790aeedf18249578889a2c1a054d0cfe951fc875bf17f93a2ad451407381eb5b/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/f8e2d22f4971b579077c2985539615deae4b4f74a07a96f073c0e2f03a47960a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/5ebafdba2e8b94091f42f368422bd7a1677e0a673a38cfe1d485362cf54b265b/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/42b03f29f1160a4ecf98da21bf7d6cad7e149b6faef9be36485f8702b8c0f285/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/fb73d6a6fb0650ec018cdb16b1897da5fe3c243cd4c08020019cbaaf37f6cd54/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/d0670935be6b57c9eea892ce1c220817f57de6893004fb9cc12b47223a11ed84/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/609285ad3d8c8ce21a2ca0ce319ec5db74f513a3ff2f16a397f31f95adc0f785/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/b87cd8f69550fda61417c30b46855a3be505513022abafe1107d431ca827be92/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/4cc14fbd4cb67fbba2420e156a8886a04889df94101b4a53ddee262aa5ab4dd7/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/f1926102881f12cd4aeadfd20a6e2929284487d7e9f53ad1c59062b014364d01/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/e7e7748b0396a6ed0130722fa9023277a057ea18bc360303e0125d8f67a19134/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/e8b8e8947a74f92969e76c1ba5345b2b5f2061791f727a44b73e2dffe15d0a38/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/6d2ba912a8b6d4afaa23d62a744205629cc6e214d8287c85408908905c0ce0cd/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/45b3e68cf38f40495b919bfd9a93e47cb0eb40bd3e953f9607c1006c1c0d4ed1/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/5fa7c7413b1e85cc3abff97e0cd294746bcec55e35906fde5da701cb46a207a7/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/f1ffffc2f3e13e0a7bb1302f77d3844717442cc9a1d1c64c93c547e58e0491bf/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/a90c62a3a04a75887696a2e86aee6585e953262724e28f5f5d0eb50e98169056/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/1e58e1a4afba8fd850728614ba6543e22d7940b17ec7754c3f1daa598e4288ce/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/c2bd411f3635e4308a463da4b222910108e954b094f98dc1f6f490df33f7063a/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/af60309087cd2238a568bdf33550086de57dfb5a13a1e4a295575843043d91e6/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/4a1ae74209c5c883f48d98b822c1e19d93baf8db4bccc7070d0611b1a66206b7/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/85a856cf29d47b317e27ea22417675a07303d6b0e8570e6c92fd0ae591f4f86d/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/3737f8b69e60a90a3b044cc9d65c5b95f43e786a3c020d6850d1e7b0fc4f21ae/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/7d2fd176650cf8d1b71087a77dd2968a6e519064ec203be9ee6c42dbe500365e/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/687f766b8525856353e816b8c199a70381eef62bb065fcd4607411f222213354/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/e884f9d20a4a1ff45b2fb047857785cbb1a1016666e0bc62087a53aa693261c4/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/a9b72def36744c403ee14534113f950b8f991f5d995172af48be8917e0b0b20d/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-341:{mountpoint:/var/lib/containers/storage/overlay/8a6051f8a285301904b261459d9c7a9bc32f7b184c2e1633a60ba1acb8ba5b5d/merged major:0 minor:341 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/ee2435af00f3168b0a38e418e3a2bce4c03e8aa2a1eee12a295b44bc0d97c578/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/eb9effd21287fb8df34dee002fd28e9f3a70a143690bc40224e5e74446392a5a/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/7fdfdef42cbd96d88250498a70d43746e031aa2435603f5fe686123780bc8f67/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/00be7c4f41483d3a49f9abd1b4d226be646eaccd74d5d820f20c750e1e1a62da/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/a4cb8d18a2bc80b536714c0a4b65c1e57a34de2997928edd549686bfbb15ed30/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/9a0c0dbd4878c4be769559f803a8a07e77c8e0d052debab94d68d049216fe9c9/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/5b9e406dc19766bf43890bb64f4d08f7d6232d94b323654f35570d83a09de342/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/6cb1334f128d032b7013c9772884d1a37d0c989b373145015bff1d9750fe220d/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/b66ed3ec3c9709ee72e22180cc8a87108e5e3cff0871dd6f1da089f403446949/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-397:{mountpoint:/var/lib/containers/storage/overlay/9eb68c2e235317b254f0f030e5f6e8c28f999bc58d69c101e3fa60af49a78f22/merged major:0 minor:397 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/48e4a2edb3b2d09f4fc2c587850ff0ccceea640384f29e3e2a63cd7358ab60fb/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/632f143484032d600f752aee708166b42ee28c632d92cc7475d37bc9775df8ba/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/026f6da4a393781416f05993c05edfc706de5864540c11e2e9809d7d91e08c92/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/901f0bdd63baa53c6ae5420cea60f64e7f89e32411ebd316caf70417fb59ee78/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/ada004bb945975125a67e4143559108c8ddebaa531062fe4c57931b624e50f13/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/ab4d84ce405fe7634c0a4d7690c2e9cfe34ea8a7613049d43bb036f25fe944c0/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/574a83805090b780bd69265b69e841af4dbf1e4d4f58c20d3a8d764cbf9239e8/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-445:{mountpoint:/var/lib/containers/storage/overlay/afcb47d8db270b6b6e3ad9bcc0a5dda84bce42658b96331048eb54c92f922668/merged major:0 minor:445 fsType:overlay blockSize:0} overlay_0-447:{mountpoint:/var/lib/containers/storage/overlay/73fcd93536e48a31771ac4565267c6873bff00910b449c1092a51ad9e0e5d074/merged major:0 minor:447 fsType:overlay blockSize:0} overlay_0-453:{mountpoint:/var/lib/containers/storage/overlay/50fe1f31d21be3faebcc3c6aee7f2ca550fb894971534fb782f9dd89faecabcc/merged major:0 minor:453 fsType:overlay blockSize:0} overlay_0-455:{mountpoint:/var/lib/containers/storage/overlay/4104f7fb2201347a5aed85d399d1605665e863d9a030137fcaf820ab24489d6f/merged major:0 minor:455 fsType:overlay blockSize:0} overlay_0-463:{mountpoint:/var/lib/containers/storage/overlay/03baa82786fa4fe09f98b74fb736c93897d99016800736cdd67ebb5f9dfbba12/merged major:0 minor:463 fsType:overlay blockSize:0} overlay_0-465:{mountpoint:/var/lib/containers/storage/overlay/83ed8b9086711836c2629de2e9c96e22e26a783c7a905effaea33a744a792ff6/merged major:0 minor:465 fsType:overlay blockSize:0} overlay_0-467:{mountpoint:/var/lib/containers/storage/overlay/38156b90b7397f01f929b3364364f91f3b7830011cb4d404ae185bf8b0ab09fc/merged major:0 minor:467 fsType:overlay blockSize:0} overlay_0-469:{mountpoint:/var/lib/containers/storage/overlay/18cca75906a2a112b909962ec06d02ab8b0772b4a1c1110b907525896e6ce1a2/merged major:0 minor:469 fsType:overlay blockSize:0} overlay_0-475:{mountpoint:/var/lib/containers/storage/overlay/8acc55e5418653828d5953771b7129d9efbe451c9d7b68a1cd9f62f29d23a71d/merged major:0 minor:475 fsType:overlay blockSize:0} overlay_0-480:{mountpoint:/var/lib/containers/storage/overlay/8dc6050c6c7f97e1c5065a7dd585c43b4329fd161227795df482c4eb722c77ca/merged major:0 minor:480 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/6210ee1496a8ac3052ba8a7ecb13584bea5f8c6ecf06669c0c4de3a90786f5e9/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-491:{mountpoint:/var/lib/containers/storage/overlay/0f28101da8ab01a38688eaf5e5ba2cdeb58e05ec41fd52cddb64ebc38ab7824c/merged major:0 minor:491 fsType:overlay blockSize:0} overlay_0-497:{mountpoint:/var/lib/containers/storage/overlay/bb36bc2a91ccbd988a08cadd72319af154e4d3444da9dba94d4743f61faa0ed4/merged major:0 minor:497 fsType:overlay blockSize:0} overlay_0-502:{mountpoint:/var/lib/containers/storage/overlay/108947d27ddf5b51ef6b0d82121885a07d791d80153d1233b2125228c4ca1bbb/merged major:0 minor:502 fsType:overlay blockSize:0} overlay_0-508:{mountpoint:/var/lib/containers/storage/overlay/4a573f9474bbe6cfefd33a4de59f3c08aa1b67b8b59b202c8b95a9df22f45765/merged major:0 minor:508 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/5bf76f61057f0aaea74d3ddb1db7e93e5f8975237f4bb6dc92a0455a01b9411b/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/dd2f60879636dfde296a41e3ea09fa74742b809148b4db6386296e3f480b96cf/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/ee3466748f34294aad09f33c78788c95a9e4da098d2225b75451221603876302/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/f0f2e88174a1c8fe2fe2568e830fe71c7970aea5dd2d78c27df2a2ebbbaedcf5/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/d8ffbe591cbe8f7c757f2678efbdf60c19b83678b21f2573532956bcda32bc39/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/05e69b0e9d04919c51536bdc0fb7acd27841fc6bf12e1c9a3662b15d8b2ce818/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5d1fcc161917e8fe426bd8bf1e1acfc0e78ea1e747cb880c7a8b223e3fb9413c/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/29cdde5733599fc0b4da36b2cbdb31b882352d7136f5bfdde3f6b0ebaf00b046/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-93:{mountpoint:/var/lib/containers/storage/overlay/b4da1b31d53df7d140cbda1e695c1555f63e048a6f104297392e19fd30728946/merged major:0 minor:93 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/76ee532e9ec18f95e137945f54ee4ca8c3bf026304ebcff51b2ed41790eeb0e9/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.623766    5915 manager.go:217] Machine: {Timestamp:2026-11-03 20:39:45.623275597 +0000 UTC m=+0.074644766 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3081357 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec291b516e41aa05dd420e29ba0208f0 SystemUUID:ec291b51-6e41-aa05-dd42-0e29ba0208f0 BootID:c08b650d-6d52-4690-b179-77a102fab2a0 Filesystems:[{Device:/run/containers/storage/overlay-containers/a74f90854ca418b10f3348c2898022bb0aec55e888957f22e54e59d83757ea17/userdata/shm DeviceMajor:0 DeviceMinor:437 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-480 DeviceMajor:0 DeviceMinor:480 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/67bbe8075f59200f3640ec5e611fd262ef28aa3e9667bfb0b1684086b5acfbc4/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1c53c625b676f8105c72b8182172e0bbd3d1ae9cba1bda928797ef7aed12940/userdata/shm DeviceMajor:0 DeviceMinor:439 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b86730e4ff9486f338d65c254ab2096a20d97bc91baeb6071aaa3f36ba927d39/userdata/shm DeviceMajor:0 DeviceMinor:441 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-508 DeviceMajor:0 DeviceMinor:508 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:210 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/kube-api-access-gbz78 DeviceMajor:0 DeviceMinor:222 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/54725514-9087-42f7-8c47-5a995d71a0ee/volumes/kubernetes.io~projected/kube-api-access-8cqjr DeviceMajor:0 DeviceMinor:228 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~projected/kube-api-access-mwzrq DeviceMajor:0 DeviceMinor:235 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32/volumes/kubernetes.io~projected/kube-api-access-jhxxb DeviceMajor:0 DeviceMinor:334 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/79e9cf5f5ac8bc0830e3f358bd60c29fd20c04693ff0305558757c7a81467038/userdata/shm DeviceMajor:0 DeviceMinor:443 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-453 DeviceMajor:0 DeviceMinor:453 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b3898fdb1c58f034ae4c825965d1a2e9b9792c898a9d76709af27325337a24a6/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-502 DeviceMajor:0 DeviceMinor:502 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~projected/kube-api-access-gf9s5 DeviceMajor:0 DeviceMinor:221 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d37aa0c-7893-407a-a52a-46d8a1ffd00e/volumes/kubernetes.io~projected/kube-api-access-s8qts DeviceMajor:0 DeviceMinor:227 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~projected/kube-api-access-v9cbl DeviceMajor:0 DeviceMinor:229 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:234 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-341 DeviceMajor:0 DeviceMinor:341 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-397 DeviceMajor:0 DeviceMinor:397 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/84298777-7d2a-4d90-80aa-0cd0c65e04f3/volumes/kubernetes.io~projected/kube-api-access-2kx9d DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~projected/kube-api-access-wrl4g DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-445 DeviceMajor:0 DeviceMinor:445 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:457 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-465 DeviceMajor:0 DeviceMinor:465 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e520b91f6e3629e7d762ab07599a64f7ad3bcea1b99ee5075aec194aa77fcc64/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1635c73de0734221b3fdd22ba9a30731b33a8d1967bce0f0ee7675c47a951cc9/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bcc39de0203dd07d98d0b8f378f1ede76c29b63513567fa156c1f12d2826811b/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~projected/kube-api-access-6fns9 DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bab0970d-6322-42d0-a66b-6d0acba24275/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:214 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46e3b51b18c0d1102293f8b39d3c6b8110f75388368a1eb6ea2bd2dd8aa8b31b/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6cccf9dc16b22cdfde78d128a11572cf60285e5e352ce10c3c978e9a17d3f838/userdata/shm DeviceMajor:0 DeviceMinor:357 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:381 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~projected/kube-api-access-r96ml DeviceMajor:0 DeviceMinor:52 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-93 DeviceMajor:0 DeviceMinor:93 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/kube-api-access-zwn5d DeviceMajor:0 DeviceMinor:232 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~projected/kube-api-access-76xpb DeviceMajor:0 DeviceMinor:233 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/40b7362ce03139caeae85b4fd65b18cf5f88bfeb3a3e1f1c9b798e63f6ec8dd8/userdata/shm DeviceMajor:0 DeviceMinor:345 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/51834fed-df6e-444b-9c2d-4501e262df12/volumes/kubernetes.io~projected/kube-api-access-lcwcr DeviceMajor:0 DeviceMinor:53 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c6d5af76-ea42-41d7-9920-db291e351c53/volumes/kubernetes.io~projected/kube-api-access-s4j8h DeviceMajor:0 DeviceMinor:54 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0/volumes/kubernetes.io~projected/kube-api-access-zwqr7 DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b2539a61c36c7797e43c92797b2db4faeb62ea000e614965b9fcb0c21450e58/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30b966c5-d3d0-4743-a041-fe5f0f196fd1/volumes/kubernetes.io~projected/kube-api-access-72v9b DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa883451821c25b2aaf580109ddfd64a5ea2d9aab4a7140e264ab2914c3c8c0d/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:230 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:47 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/66a0afd2-dcb3-48ab-905d-4e4abab2fe97/volumes/kubernetes.io~projected/kube-api-access-hwvc7 DeviceMajor:0 DeviceMinor:51 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9665ed4d53a387ab34d34c47fad64eef444c35118c37957f277c81d130812a89/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/268514945afbd8aa706a3d4f02ce4603a82d67dde033ad4bd680eea123b9937b/userdata/shm DeviceMajor:0 DeviceMinor:386 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:408 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0d4c7d4b3d1fdcc3cbd33c4e31c25e4e493fa190dc32ea8fe150e823b253171c/userdata/shm DeviceMajor:0 DeviceMinor:413 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f76fe3d1-1a83-4e2b-867c-792eba90309f/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls DeviceMajor:0 DeviceMinor:436 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-467 DeviceMajor:0 DeviceMinor:467 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-463 DeviceMajor:0 DeviceMinor:463 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:48 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/476f59f4-3347-4e66-b0ea-51add44cfc27/volumes/kubernetes.io~projected/kube-api-access-f46q7 DeviceMajor:0 DeviceMinor:286 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d/volumes/kubernetes.io~secret/samples-operator-tls DeviceMajor:0 DeviceMinor:430 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5aba65ad626461927b921daefb08b304a70d8a307b1e85001bb907fe712c4ce0/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~projected/kube-api-access-djxhv DeviceMajor:0 DeviceMinor:356 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-469 DeviceMajor:0 DeviceMinor:469 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-475 DeviceMajor:0 DeviceMinor:475 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/90155d8a-eff2-4b22-ac1b-975bffe1d644/volumes/kubernetes.io~projected/kube-api-access-47h9w DeviceMajor:0 DeviceMinor:205 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2970a76f752d2c0446811c8116b1061a059e0d1b150ea71f904ff743e8a1c76f/userdata/shm DeviceMajor:0 DeviceMinor:223 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~projected/kube-api-access-2ggk7 DeviceMajor:0 DeviceMinor:226 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5c92695d378a43c8d841ce5f8f23e24c7e07287b695ec1a9aa2c19b3019e3a45/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8e590e6f-ebec-4c39-8f00-d9460e3aeb57/volumes/kubernetes.io~projected/kube-api-access-kj8wv DeviceMajor:0 DeviceMinor:337 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe/volumes/kubernetes.io~projected/kube-api-access-ssmxp DeviceMajor:0 DeviceMinor:340 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b258928c-3513-426a-b50b-dd0f29f24a21/volumes/kubernetes.io~secret/networking-console-plugin-cert DeviceMajor:0 DeviceMinor:435 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-122 DeviceMajor:0 DeviceMinor:122 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e7babdeb287a7093e13d1d2e44e77ebfbc0060d6969ab40a5b861b1451339613/userdata/shm DeviceMajor:0 DeviceMinor:459 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/83806e59-a55b-4491-b6d0-59b373bac613/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f1124bb22307af3c99f69c44f3274cdb98b0da252e3667f07db4158dc357a152/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97/volumes/kubernetes.io~projected/kube-api-access-kp5qz DeviceMajor:0 DeviceMinor:329 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:371 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb/volumes/kubernetes.io~projected/kube-api-access-vlhzq DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bb233d06-f91f-4db9-be7b-bb9966b67a75/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:50 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e0a36ab34224b3f3823607e823a8fcf48a738b26ce3c80e4a22cdb5d458bc599/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:211 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-108 DeviceMajor:0 DeviceMinor:108 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91453c10-6f59-4fd5-bf2a-69c2c3944375/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:213 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6c893222-ab83-4a89-98e2-169613906e67/volumes/kubernetes.io~projected/kube-api-access-xvrs8 DeviceMajor:0 DeviceMinor:231 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-455 DeviceMajor:0 DeviceMinor:455 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4ad76d3e4090f29aecd1714ab85e5a8cfbd8603471c1c5a0beb6d78e88351340/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-491 DeviceMajor:0 DeviceMinor:491 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1c22090d-43fa-4fa7-adac-ecff9757f988/volumes/kubernetes.io~projected/kube-api-access-hxkq8 DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/99a1c8abd9ab152bd056f95c1b10ed0228b9a74ae291e7beab9e100dbb06e39a/userdata/shm DeviceMajor:0 DeviceMinor:338 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87fef4f1-d40d-4a12-9e0f-371838f9ecfa/volumes/kubernetes.io~projected/kube-api-access-bbjdn DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e62edc4-aa68-49ad-ae64-909e88517f0f/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:355 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d0be0eb53bdfc499b733147c272eab60f424cd9fa4cbd5ce7e45622e32f01791/userdata/shm DeviceMajor:0 DeviceMinor:460 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9/userdata/shm DeviceMajor:0 DeviceMinor:384 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5713f5a0-c8bf-44c1-aeff-c9c580746c45/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:458 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-497 DeviceMajor:0 DeviceMinor:497 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/badb02200e7b6a26159ae329e8c2aa083c02bac0dfd4cbe5ba70c80949016727/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:372 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516/userdata/shm DeviceMajor:0 DeviceMinor:382 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ff27c4bf-c1d8-47f8-9737-24510e201615/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:49 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb6cc3a8571205d1179ee37712ec94085c9ac2047a1f3a17a78e63d1097517d2/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/60bfa38c-92bc-4bef-ac05-18410621ea88/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:434 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-447 DeviceMajor:0 DeviceMinor:447 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b768b17b-8769-4002-8c89-3bee5fda2060/volumes/kubernetes.io~projected/kube-api-access-979r2 DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4e010e9ff98161cb7c95fa26a6292bfa2b83c5e73d52eba56245c2b2b313bdd/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1a4ba319a61a48f80794624af1737285205bed3a3cfaa4520c5abfca3cc82659/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4c2480e6-eaa3-402a-b3fc-2c63f26ff172/volumes/kubernetes.io~projected/kube-api-access-4cr8r DeviceMajor:0 DeviceMinor:206 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:212 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0d4c7d4b3d1fdcc MacAddress:16:51:32:01:95:1f Speed:10000 Mtu:8901} {Name:268514945afbd8a MacAddress:9e:3c:56:35:dd:af Speed:10000 Mtu:8901} {Name:2970a76f752d2c0 MacAddress:9a:90:6d:6c:5f:36 Speed:10000 Mtu:8901} {Name:40b7362ce03139c MacAddress:de:f9:4c:ec:69:fc Speed:10000 Mtu:8901} {Name:46e3b51b18c0d11 MacAddress:0e:a3:32:44:d5:71 Speed:10000 Mtu:8901} {Name:4728685f325537d MacAddress:92:98:90:15:a9:c2 Speed:10000 Mtu:8901} {Name:4ad76d3e4090f29 MacAddress:8e:79:74:fe:54:33 Speed:10000 Mtu:8901} {Name:5aba65ad6264619 MacAddress:92:9f:c3:d0:cd:ad Speed:10000 Mtu:8901} {Name:5c92695d378a43c MacAddress:f2:0f:20:63:99:ef Speed:10000 Mtu:8901} {Name:67bbe8075f59200 MacAddress:8e:09:27:d3:5f:8a Speed:10000 Mtu:8901} {Name:6cccf9dc16b22cd MacAddress:ea:ad:de:37:0b:b8 Speed:10000 Mtu:8901} {Name:79e9cf5f5ac8bc0 MacAddress:b2:16:6e:e0:61:c5 Speed:10000 Mtu:8901} {Name:99a1c8abd9ab152 MacAddress:82:19:da:d9:6b:96 Speed:10000 Mtu:8901} {Name:a1c53c625b676f8 MacAddress:66:4a:6d:02:9a:72 Speed:10000 Mtu:8901} {Name:a74f90854ca418b MacAddress:1a:b5:7c:1f:63:45 Speed:10000 Mtu:8901} {Name:ab97a79061ca803 MacAddress:b6:dc:0c:8b:7d:3e Speed:10000 Mtu:8901} {Name:b3898fdb1c58f03 MacAddress:ca:30:72:3e:0a:81 Speed:10000 Mtu:8901} {Name:b86730e4ff9486f MacAddress:5e:bf:72:ca:58:62 Speed:10000 Mtu:8901} {Name:bcc39de0203dd07 MacAddress:fe:9b:da:5b:7b:bb Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:15:05:d7:82:e1 Speed:0 Mtu:9001} {Name:br-int MacAddress:6e:07:41:53:99:09 Speed:0 Mtu:8901} {Name:c4e010e9ff98161 MacAddress:b2:d9:3b:e0:a9:47 Speed:10000 Mtu:8901} {Name:d0be0eb53bdfc49 MacAddress:02:04:ae:ee:21:69 Speed:10000 Mtu:8901} {Name:e15b1e950f36eed MacAddress:3e:7e:60:a9:8e:1f Speed:10000 Mtu:8901} {Name:e7babdeb287a709 MacAddress:2a:2f:b0:e3:31:5a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:15:05:d7:82:e1 Speed:0 Mtu:9001} {Name:f1124bb22307af3 MacAddress:6e:d5:15:09:a9:2d Speed:10000 Mtu:8901} {Name:fb6cc3a8571205d MacAddress:ea:3f:58:0e:83:cb Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:06:68:e5:49:7a:83 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:da:a7:a4:04:58:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624192    5915 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624234    5915 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624490    5915 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624512    5915 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-248.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624672    5915 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624679    5915 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624689    5915 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624702    5915 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624720    5915 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 20:39:45.624913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624871    5915 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.624963    5915 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625006    5915 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625015    5915 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625025    5915 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625033    5915 kubelet.go:405] "Adding apiserver pod source"
Nov 03 20:39:45.625417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625040    5915 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 20:39:45.625808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625795    5915 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 20:39:45.625849 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.625811    5915 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:39:45.626298 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.626280    5915 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 20:39:45.629765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.629750    5915 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 20:39:45.629765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.629764    5915 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:39:45.629885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.629875    5915 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 20:39:45.630095 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630084    5915 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 20:39:45.630134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630114    5915 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 20:39:45.630224 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630215    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 20:39:45.630254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630229    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 20:39:45.630254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630235    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 20:39:45.630254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630240    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 20:39:45.630254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630246    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 20:39:45.630254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630252    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630258    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630267    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630274    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630279    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630291    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630300    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630326    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 20:39:45.630422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.630336    5915 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 20:39:45.633823 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.633812    5915 server.go:1263] "Started kubelet"
Nov 03 20:39:45.633898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.633877    5915 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 20:39:45.634472 ip-10-0-141-248 systemd[1]: Started Kubernetes Kubelet.
Nov 03 20:39:45.635161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.633914    5915 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 20:39:45.635161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.634795    5915 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 20:39:45.635161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.635109    5915 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 20:39:45.637736 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.636825    5915 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.640959    5915 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.640997    5915 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.641027    5915 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.641041    5915 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.641066    5915 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:35 +0000 UTC" deadline="2028-07-11 21:26:04.962505028 +0000 UTC"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.641086    5915 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14784h46m19.321422414s"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.641141    5915 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 20:39:45.641492 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.641122    5915 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-248.us-west-1.compute.internal\" not found"
Nov 03 20:39:45.644351 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644320    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bab0970d-6322-42d0-a66b-6d0acba24275" volumeName="kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644353    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f76fe3d1-1a83-4e2b-867c-792eba90309f" volumeName="kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644368    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644381    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="476f59f4-3347-4e66-b0ea-51add44cfc27" volumeName="kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644392    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60bfa38c-92bc-4bef-ac05-18410621ea88" volumeName="kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644403    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84298777-7d2a-4d90-80aa-0cd0c65e04f3" volumeName="kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644415    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" volumeName="kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644426    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c2480e6-eaa3-402a-b3fc-2c63f26ff172" volumeName="kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth" seLinuxMountContext=""
Nov 03 20:39:45.644442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644442    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c893222-ab83-4a89-98e2-169613906e67" volumeName="kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644453    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe" volumeName="kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644465    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" volumeName="kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644478    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb233d06-f91f-4db9-be7b-bb9966b67a75" volumeName="kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644489    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644501    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60bfa38c-92bc-4bef-ac05-18410621ea88" volumeName="kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644516    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91453c10-6f59-4fd5-bf2a-69c2c3944375" volumeName="kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644528    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" volumeName="kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644539    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e62edc4-aa68-49ad-ae64-909e88517f0f" volumeName="kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644551    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97" volumeName="kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644580    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6d5af76-ea42-41d7-9920-db291e351c53" volumeName="kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644592    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644621    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6d5af76-ea42-41d7-9920-db291e351c53" volumeName="kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644633    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644645    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644657    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c2480e6-eaa3-402a-b3fc-2c63f26ff172" volumeName="kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644668    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d" volumeName="kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644680    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" volumeName="kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644692    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83806e59-a55b-4491-b6d0-59b373bac613" volumeName="kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644709    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84298777-7d2a-4d90-80aa-0cd0c65e04f3" volumeName="kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644732    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90155d8a-eff2-4b22-ac1b-975bffe1d644" volumeName="kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644745    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c2480e6-eaa3-402a-b3fc-2c63f26ff172" volumeName="kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644757    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" volumeName="kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644769    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b768b17b-8769-4002-8c89-3bee5fda2060" volumeName="kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644782    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bab0970d-6322-42d0-a66b-6d0acba24275" volumeName="kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644794    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb233d06-f91f-4db9-be7b-bb9966b67a75" volumeName="kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644806    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6d5af76-ea42-41d7-9920-db291e351c53" volumeName="kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644826    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644837    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644849    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644861    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644873    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle" seLinuxMountContext=""
Nov 03 20:39:45.644869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644885    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66a0afd2-dcb3-48ab-905d-4e4abab2fe97" volumeName="kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644896    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644908    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644920    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97" volumeName="kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644932    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97" volumeName="kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644944    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb233d06-f91f-4db9-be7b-bb9966b67a75" volumeName="kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644957    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644970    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644983    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5084c67b-30c3-4ee8-a72c-a2f76b1c8c64" volumeName="kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.644994    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60bfa38c-92bc-4bef-ac05-18410621ea88" volumeName="kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645006    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b258928c-3513-426a-b50b-dd0f29f24a21" volumeName="kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645019    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6d5af76-ea42-41d7-9920-db291e351c53" volumeName="kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645030    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff27c4bf-c1d8-47f8-9737-24510e201615" volumeName="kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645042    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645053    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645066    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c2480e6-eaa3-402a-b3fc-2c63f26ff172" volumeName="kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645087    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5713f5a0-c8bf-44c1-aeff-c9c580746c45" volumeName="kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645099    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f8781a-81f9-43f9-bd78-c5bcf5191fc0" volumeName="kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645109    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645120    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54725514-9087-42f7-8c47-5a995d71a0ee" volumeName="kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645132    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66a0afd2-dcb3-48ab-905d-4e4abab2fe97" volumeName="kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645144    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e62edc4-aa68-49ad-ae64-909e88517f0f" volumeName="kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645156    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e62edc4-aa68-49ad-ae64-909e88517f0f" volumeName="kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645160    5915 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645167    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f8781a-81f9-43f9-bd78-c5bcf5191fc0" volumeName="kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645176    5915 factory.go:55] Registering systemd factory
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645179    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4f8781a-81f9-43f9-bd78-c5bcf5191fc0" volumeName="kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645186    5915 factory.go:223] Registration of the systemd container factory successfully
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645191    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97" volumeName="kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645204    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645215    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1747a6b1-0e38-4948-baa2-a75cbe265752" volumeName="kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645228    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e5e0dcf-8842-4b4a-bc4b-d11e653865fb" volumeName="kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645239    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5084c67b-30c3-4ee8-a72c-a2f76b1c8c64" volumeName="kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645251    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51834fed-df6e-444b-9c2d-4501e262df12" volumeName="kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645263    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d" volumeName="kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645275    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83806e59-a55b-4491-b6d0-59b373bac613" volumeName="kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645290    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87fef4f1-d40d-4a12-9e0f-371838f9ecfa" volumeName="kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645302    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645315    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5084c67b-30c3-4ee8-a72c-a2f76b1c8c64" volumeName="kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645331    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" volumeName="kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645342    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b768b17b-8769-4002-8c89-3bee5fda2060" volumeName="kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645353    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f76fe3d1-1a83-4e2b-867c-792eba90309f" volumeName="kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645364    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ff27c4bf-c1d8-47f8-9737-24510e201615" volumeName="kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645378    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0f528bfd-d20d-4fe1-81ae-754987ca453f" volumeName="kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645389    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30b966c5-d3d0-4743-a041-fe5f0f196fd1" volumeName="kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645401    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5084c67b-30c3-4ee8-a72c-a2f76b1c8c64" volumeName="kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645415    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5084c67b-30c3-4ee8-a72c-a2f76b1c8c64" volumeName="kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645430    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60bfa38c-92bc-4bef-ac05-18410621ea88" volumeName="kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645445    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87fef4f1-d40d-4a12-9e0f-371838f9ecfa" volumeName="kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645457    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f76fe3d1-1a83-4e2b-867c-792eba90309f" volumeName="kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645469    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1c22090d-43fa-4fa7-adac-ecff9757f988" volumeName="kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645482    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" volumeName="kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645513    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" volumeName="kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645524    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83806e59-a55b-4491-b6d0-59b373bac613" volumeName="kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645536    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e590e6f-ebec-4c39-8f00-d9460e3aeb57" volumeName="kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645548    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91453c10-6f59-4fd5-bf2a-69c2c3944375" volumeName="kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645552    5915 factory.go:153] Registering CRI-O factory
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645577    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b258928c-3513-426a-b50b-dd0f29f24a21" volumeName="kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645587    5915 factory.go:223] Registration of the crio container factory successfully
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645590    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bab0970d-6322-42d0-a66b-6d0acba24275" volumeName="kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g" seLinuxMountContext=""
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645605    5915 factory.go:103] Registering Raw factory
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645620    5915 manager.go:1199] Started watching for new ooms in manager
Nov 03 20:39:45.646064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645950    5915 manager.go:319] Starting recovery of all containers
Nov 03 20:39:45.648787 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.645601    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c2480e6-eaa3-402a-b3fc-2c63f26ff172" volumeName="kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs" seLinuxMountContext=""
Nov 03 20:39:45.648830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.648808    5915 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d" volumeName="kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5" seLinuxMountContext=""
Nov 03 20:39:45.648866 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.648829    5915 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 20:39:45.648866 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.648838    5915 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 20:39:45.659814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.659791    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:45.660905 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.660886    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:45.671679 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.671629    5915 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 20:39:45.673198 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.673178    5915 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 20:39:45.673198 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.673197    5915 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 20:39:45.673338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.673219    5915 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 20:39:45.673338 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.673255    5915 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 20:39:45.685552 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.685527    5915 generic.go:356] "Generic (PLEG): container finished" podID="a713afcbd6eef9b8d78d253711a83e69" containerID="d82c6d0e520ceea9178599028b00a575ec79ed17577c6a78569d4dda72733bca" exitCode=0
Nov 03 20:39:45.709863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709837    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="be653610dbc6aedc11034b285763b51780eedeed94507129cdef0ce9618d5c9c" exitCode=0
Nov 03 20:39:45.709863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709859    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="8a236a7de2d972502bed28799ec19e3e7446e400bddd1e11d464c66bfc5ecaab" exitCode=0
Nov 03 20:39:45.709863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709866    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="0a782479dafd2cfbc6f375ef2460cf8f96d2ef1aea4234b62e18f832125907db" exitCode=0
Nov 03 20:39:45.710105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709872    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="0ab97e058bf8a288ebc3a953bd344421dcd7af67d0e4c41caaa4aad087037915" exitCode=0
Nov 03 20:39:45.710105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709878    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="f682c0c5c8af87e7c68e54389bd1dee0f3a2f5889b6da9b008198fd08a24b31c" exitCode=0
Nov 03 20:39:45.710105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.709883    5915 generic.go:356] "Generic (PLEG): container finished" podID="c6d5af76-ea42-41d7-9920-db291e351c53" containerID="fcf6bcbbb4070df8d6f605487becc08e0efc922a37d26aa8ef820203f212ee56" exitCode=0
Nov 03 20:39:45.741421 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.741329    5915 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-248.us-west-1.compute.internal\" not found"
Nov 03 20:39:45.744791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.744760    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:45.744866 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.744795    5915 generic.go:356] "Generic (PLEG): container finished" podID="6d37aa0c-7893-407a-a52a-46d8a1ffd00e" containerID="3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895" exitCode=255
Nov 03 20:39:45.765869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.765850    5915 manager.go:324] Recovery completed
Nov 03 20:39:45.773315 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.773297    5915 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 03 20:39:45.788189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.788167    5915 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:45.789711 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.789688    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:45.789791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.789718    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:45.789791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.789731    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:45.792714 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792697    5915 cpu_manager.go:226] "Starting" policy="none"
Nov 03 20:39:45.792714 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792710    5915 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 20:39:45.792714 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792714    5915 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:45.792894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792729    5915 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 20:39:45.792894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792841    5915 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 20:39:45.792894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792851    5915 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 20:39:45.793031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792973    5915 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 20:39:45.793031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792979    5915 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 20:39:45.793031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792984    5915 policy_none.go:50] "Start"
Nov 03 20:39:45.793031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.792993    5915 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 20:39:45.793031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.793002    5915 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 20:39:45.793539 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.793526    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:45.793619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.793550    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:45.793619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.793578    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:45.798669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.798656    5915 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 20:39:45.798669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.798670    5915 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 20:39:45.798751 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.798676    5915 policy_none.go:44] "Start"
Nov 03 20:39:45.803459 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.803436    5915 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 20:39:45.805974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.805961    5915 manager.go:341] "Starting Device Plugin manager"
Nov 03 20:39:45.806048 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.805990    5915 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 20:39:45.806048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806001    5915 server.go:87] "Starting device plugin registration server"
Nov 03 20:39:45.806311 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806297    5915 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 20:39:45.806382 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806310    5915 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 20:39:45.806438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806396    5915 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 20:39:45.806531 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806495    5915 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 20:39:45.806619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806545    5915 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 20:39:45.806619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806551    5915 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 20:39:45.806721 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.806620    5915 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T20:39:45.806530115Z","UUID":"3e83328e-5f3e-4711-90cf-47210a86d442","Handler":null,"Name":"","Endpoint":""}
Nov 03 20:39:45.808290 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.808274    5915 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 20:39:45.808368 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:45.808310    5915 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-248.us-west-1.compute.internal\" not found"
Nov 03 20:39:45.808896 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.808879    5915 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 20:39:45.808960 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.808906    5915 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 20:39:45.837075 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.837052    5915 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:45.837160 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.837142    5915 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 20:39:45.837773 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.837756    5915 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:45.837890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.837876    5915 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:45.888227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.888202    5915 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:45.907134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.907113    5915 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:45.908055 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.908040    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:45.908133 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.908067    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:45.908133 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.908078    5915 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-248.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:45.908133 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.908118    5915 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:45.973814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.973774    5915 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"]
Nov 03 20:39:45.974352 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974322    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerStarted","Data":"0a380862285ebe8b8fb2e55ca3b9d20324602747d28b1b56de270e4bb1a91604"}
Nov 03 20:39:45.974453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974358    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerDied","Data":"d82c6d0e520ceea9178599028b00a575ec79ed17577c6a78569d4dda72733bca"}
Nov 03 20:39:45.974453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974373    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"a713afcbd6eef9b8d78d253711a83e69","Type":"ContainerStarted","Data":"943a2256207bd5573a637386da1898c147d7d4926d89a96e1cb54dd63eccd9cd"}
Nov 03 20:39:45.974453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974384    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"fd3e66eb7ec8c0db50f85619df6599ce","Type":"ContainerStarted","Data":"ed8ad571d6aa8d42b02367f727246458dfae63f28a53762af719d61e3360951a"}
Nov 03 20:39:45.974453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974395    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" event={"ID":"fd3e66eb7ec8c0db50f85619df6599ce","Type":"ContainerStarted","Data":"69c498677611e5fd97ba67b9d0b9debc3fb90d28c9f5d5d73795628e5cc8d8a0"}
Nov 03 20:39:45.974741 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974506    5915 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:45.974741 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:45.974518    5915 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.035043 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.034992    5915 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:46.035043 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:46.035029    5915 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.035635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.035624    5915 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:46.035697 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:46.035645    5915 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.043100 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.043085    5915 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.043162 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.043136    5915 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.138165 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.138137    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.138280 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.138182    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.138280 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.138203    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238488    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238523    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238548    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238621    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd3e66eb7ec8c0db50f85619df6599ce-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"fd3e66eb7ec8c0db50f85619df6599ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238623    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.238676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.238621    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a713afcbd6eef9b8d78d253711a83e69-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" (UID: \"a713afcbd6eef9b8d78d253711a83e69\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.625534 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.625505    5915 apiserver.go:52] "Watching apiserver"
Nov 03 20:39:46.729784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.729761    5915 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 20:39:46.730479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.730455    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj","openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n","openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w","kube-system/konnectivity-agent-cks2d","openshift-image-registry/node-ca-4cjfz","openshift-cluster-node-tuning-operator/tuned-62lhn","openshift-insights/insights-runtime-extractor-tph4g","openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f","kube-system/global-pull-secret-syncer-xjbt9","openshift-insights/insights-operator-7c4986d449-zrjt5","openshift-insights/periodic-gathering-lbpvw-klqsm","openshift-console-operator/console-operator-7cd56b8dbc-xv2zb","kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-sl28h","openshift-multus/multus-b8zc5","openshift-network-console/networking-console-plugin-86c478bb88-4qvq7","openshift-multus/network-metrics-daemon-ffxrp","openshift-image-registry/image-registry-5cf6c99f88-pj2bd","openshift-network-operator/iptables-alerter-c8m6k","openshift-dns/node-resolver-66w89","kube-system/kas-connection-checker-685895c7c9-w4kw4","openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph","openshift-ingress/router-default-5769c487dc-z9pqs","kube-system/kas-connection-checker-685895c7c9-l9wgt","openshift-dns/dns-default-zjglm","openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx","openshift-multus/multus-additional-cni-plugins-vc9ll","openshift-image-registry/image-registry-98474cc5c-vc78r","openshift-ingress-canary/ingress-canary-6sv74","kube-system/kas-connection-checker-685895c7c9-dnb6d","openshift-service-ca/service-ca-5fffc97bdf-r2rmg","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz","openshift-ovn-kubernetes/ovnkube-node-gft95","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"]
Nov 03 20:39:46.750937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.750916    5915 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.751195 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.751178    5915 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.785159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785094    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.785159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785119    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.785159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785097    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 03 20:39:46.785159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785150    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.785455 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785237    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.785455 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785247    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 03 20:39:46.785455 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785287    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.785455 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.785361    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.788020 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.788003    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 03 20:39:46.840533 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.840510    5915 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:46.840685 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:46.840578    5915 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.841275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.841261    5915 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:46.841349 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:39:46.841300    5915 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.us-west-1.compute.internal"
Nov 03 20:39:46.842250 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842232    5915 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 20:39:46.842572 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842539    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:46.842650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842579    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrl4g\" (UniqueName: \"kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.842650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842597    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.842650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842622    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.842650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842639    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.842801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842654    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.842801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842712    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcwcr\" (UniqueName: \"kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.842801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842750    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.842801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842775    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwzrq\" (UniqueName: \"kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842801    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842817    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842840    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842865    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842890    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842916    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842940    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47h9w\" (UniqueName: \"kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w\") pod \"kas-connection-checker-685895c7c9-l9wgt\" (UID: \"90155d8a-eff2-4b22-ac1b-975bffe1d644\") " pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:46.842964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842965    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fns9\" (UniqueName: \"kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.842982    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843020    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843058    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843096    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843130    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843147    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843165    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843201    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjdn\" (UniqueName: \"kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.843264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843226    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843285    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72v9b\" (UniqueName: \"kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b\") pod \"kas-connection-checker-685895c7c9-dnb6d\" (UID: \"30b966c5-d3d0-4743-a041-fe5f0f196fd1\") " pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843347    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843475    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843518    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843550    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843584    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843608    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqjr\" (UniqueName: \"kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr\") pod \"network-check-source-6f78d59f99-f6gzh\" (UID: \"54725514-9087-42f7-8c47-5a995d71a0ee\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:46.843650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843641    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843678    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843707    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843733    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843778    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xpb\" (UniqueName: \"kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843888    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843921    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-tuned\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.843955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843922    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.843968    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844009    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844066    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844102    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844165    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844209    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxkq8\" (UniqueName: \"kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844247    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.844256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844250    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bab0970d-6322-42d0-a66b-6d0acba24275-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844269    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djxhv\" (UniqueName: \"kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844334    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844366    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844461    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844533    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp5qz\" (UniqueName: \"kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844551    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844522    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84298777-7d2a-4d90-80aa-0cd0c65e04f3-tmp-dir\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.844630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844596    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.844928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844638    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.844928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844701    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.844928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844757    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.844928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844726    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-tmp\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.844928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844903    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.845085 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844940    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ggk7\" (UniqueName: \"kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:46.845085 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.844981    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.845085 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845010    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979r2\" (UniqueName: \"kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:46.845085 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845028    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.845221 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845092    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.845221 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845139    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.845221 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845180    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.845364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845215    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.845364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845272    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cbl\" (UniqueName: \"kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.845364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845295    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-data-volume\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.845519 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845397    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.845519 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845436    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.845637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845519    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvc7\" (UniqueName: \"kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.845637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845578    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj8wv\" (UniqueName: \"kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv\") pod \"migrator-66df5d496b-wjr2w\" (UID: \"8e590e6f-ebec-4c39-8f00-d9460e3aeb57\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:46.845637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845617    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.845799 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845663    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8qts\" (UniqueName: \"kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.845799 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845713    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.845872 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845811    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4j8h\" (UniqueName: \"kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.845924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845874    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.845924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845905    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.846023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845932    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.845977    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846007    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.846158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846065    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.846158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846091    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.846158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846133    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.846274 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846162    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.846274 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846225    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.846343 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846311    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846405 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846386    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.846458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846439    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.846512 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846476    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.846583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846516    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.846630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846551    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c22090d-43fa-4fa7-adac-ecff9757f988-snapshots\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.846630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846587    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kx9d\" (UniqueName: \"kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.846630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846618    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.846797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846651    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.846797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846670    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.846797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846672    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.846797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846708    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.846797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846796    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846854    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846902    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqr7\" (UniqueName: \"kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.846999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846938    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:46.846999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.846978    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.847132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847063    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cr8r\" (UniqueName: \"kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.847132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847084    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.847217 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847143    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.847262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847240    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.847330 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847290    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.847374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847350    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.847410 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847385    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.847446 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847411    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.847446 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847432    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.847513 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847477    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.847615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847592    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.847659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847635    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssmxp\" (UniqueName: \"kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp\") pod \"migrator-66df5d496b-9zp4f\" (UID: \"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:46.847700 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847661    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.847700 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847674    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bab0970d-6322-42d0-a66b-6d0acba24275-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.847788 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847690    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.847837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847802    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.847837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847807    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.847943 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847889    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96ml\" (UniqueName: \"kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.847943 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847918    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.848017 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847955    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.848017 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.847999    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.848105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848077    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.848161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848129    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.848205 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848159    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.848205 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848200    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.848299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848285    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.848350 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848314    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.848457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848350    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.848457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848393    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.848457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848413    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60bfa38c-92bc-4bef-ac05-18410621ea88-tmp-dir\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.848457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848420    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.848457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848455    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlhzq\" (UniqueName: \"kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq\") pod \"kas-connection-checker-685895c7c9-w4kw4\" (UID: \"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb\") " pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:46.848718 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848488    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.848718 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848548    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.848718 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848639    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.848830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848730    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.848830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848775    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:46.848960 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.848941    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrs8\" (UniqueName: \"kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8\") pod \"volume-data-source-validator-58c9f5f46d-v6ssz\" (UID: \"6c893222-ab83-4a89-98e2-169613906e67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:46.849038 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849018    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.849072 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849031    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f76fe3d1-1a83-4e2b-867c-792eba90309f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.849072 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849067    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:39:46.849142 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849119    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.849180 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849163    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.849228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849201    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.849275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849247    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.849713 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849697    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.849769 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849751    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb233d06-f91f-4db9-be7b-bb9966b67a75-tmp\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.849809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849777    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.849809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849806    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.849887 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849831    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.849937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849896    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.849992 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849930    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.849992 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849968    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9s5\" (UniqueName: \"kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.850061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.849996    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.850096 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.850070    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.850145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.850101    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.850191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.850151    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.889003 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.888943    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.889097 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889062    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 20:39:46.889176 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889163    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 03 20:39:46.889279 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889267    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.889375 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889363    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 03 20:39:46.889486 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889473    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 03 20:39:46.889702 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889640    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.889777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889723    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-cabundle\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.889777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889760    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-default-certificate\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.889887 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889808    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.890035 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889930    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 20:39:46.890035 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889947    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890035 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889964    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890035 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889986    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:46.890263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890072    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890111    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nnzhp\""
Nov 03 20:39:46.890263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890183    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.890451 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890266    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890338    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6k78\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890482    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zvv9n\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890507    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890522    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fj5k\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890341    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890551    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890575    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 03 20:39:46.890576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890365    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890597    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890399    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890691    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890737    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890758    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890764    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890829    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890874    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890895    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 20:39:46.890957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889993    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890901    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890349    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891069    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vnlw8\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891094    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891188    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891225    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 03 20:39:46.891262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.890356    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891147    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891354    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p5s6h\""
Nov 03 20:39:46.891543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891440    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891439    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891535    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891538    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891226    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891637    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891675    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891689    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891723    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891735    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891748    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 03 20:39:46.891784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891772    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 03 20:39:46.892067 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.891801    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 20:39:46.892067 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.889812    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-kqp9s\""
Nov 03 20:39:46.892314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892289    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.892380 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892337    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm4wl\""
Nov 03 20:39:46.892380 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892350    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nf42\""
Nov 03 20:39:46.892380 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892379    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xfms\""
Nov 03 20:39:46.892519 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892432    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 03 20:39:46.892519 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892506    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892509    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892526    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2v8j5\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892543    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892549    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892548    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxrmv\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892588    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 20:39:46.892654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892613    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892704    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892771    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fjpbj\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892830    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892858    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892898    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892911    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892949    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 20:39:46.892977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.892956    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893003    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-env-overrides\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893027    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-29dtl\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893047    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893084    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893108    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-czwwq\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893128    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893048    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-plrnn\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893134    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893173    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893191    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9ll5z\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893202    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893210    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893225    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893285    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 03 20:39:46.893338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893312    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893398    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893429    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b768b17b-8769-4002-8c89-3bee5fda2060-metrics-certs\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893452    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-serving-cert\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893494    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893572    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893660    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 20:39:46.893850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.893701    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff27c4bf-c1d8-47f8-9737-24510e201615-konnectivity-ca\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:39:46.894099 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894080    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-service-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.894285 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894266    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.894337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894323    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-iptables-alerter-script\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.894532 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894518    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.894825 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894809    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b258928c-3513-426a-b50b-dd0f29f24a21-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:46.894898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894850    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-config\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.895001 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.894985    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/60bfa38c-92bc-4bef-ac05-18410621ea88-metrics-tls\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.895640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.895620    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/60bfa38c-92bc-4bef-ac05-18410621ea88-config-volume\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.896033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896016    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.896150 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896137    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5713f5a0-c8bf-44c1-aeff-c9c580746c45-original-pull-secret\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.896210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896180    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.896210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896192    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-service-ca-bundle\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.896275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896266    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.896827 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896809    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/83806e59-a55b-4491-b6d0-59b373bac613-config\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.896906 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896834    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-daemon-config\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.896958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896924    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-cni-binary-copy\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.896958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.896939    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9e62edc4-aa68-49ad-ae64-909e88517f0f-signing-key\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.898104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898089    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-metrics-certs\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.898172 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898118    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovn-node-metrics-cert\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.898230 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898216    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c22090d-43fa-4fa7-adac-ecff9757f988-serving-cert\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.898334 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898319    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.898410 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898394    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cni-binary-copy\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.898500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898488    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/83806e59-a55b-4491-b6d0-59b373bac613-serving-cert\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.898920 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.898905    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-serviceca\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.899028 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.899015    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.899063 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.899049    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-config\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.899140 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.899126    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91453c10-6f59-4fd5-bf2a-69c2c3944375-cert\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:46.900265 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.900249    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.900323 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.900267    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-stats-auth\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.900323 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.900292    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff27c4bf-c1d8-47f8-9737-24510e201615-agent-certs\") pod \"konnectivity-agent-cks2d\" (UID: \"ff27c4bf-c1d8-47f8-9737-24510e201615\") " pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:39:46.900658 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.900641    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.903533 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.903513    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c6d5af76-ea42-41d7-9920-db291e351c53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.903611 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.903532    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.903611 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.903603    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f76fe3d1-1a83-4e2b-867c-792eba90309f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.903682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.903640    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b258928c-3513-426a-b50b-dd0f29f24a21-nginx-conf\") pod \"networking-console-plugin-86c478bb88-4qvq7\" (UID: \"b258928c-3513-426a-b50b-dd0f29f24a21\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-4qvq7"
Nov 03 20:39:46.936458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.936444    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 20:39:46.937298 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.937265    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-ovnkube-script-lib\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.938669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.938640    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 03 20:39:46.938767 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.938713    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 03 20:39:46.939271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939190    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrl4g\" (UniqueName: \"kubernetes.io/projected/bab0970d-6322-42d0-a66b-6d0acba24275-kube-api-access-wrl4g\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx\" (UID: \"bab0970d-6322-42d0-a66b-6d0acba24275\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx"
Nov 03 20:39:46.939354 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939338    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcwcr\" (UniqueName: \"kubernetes.io/projected/51834fed-df6e-444b-9c2d-4501e262df12-kube-api-access-lcwcr\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.939470 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939443    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72v9b\" (UniqueName: \"kubernetes.io/projected/30b966c5-d3d0-4743-a041-fe5f0f196fd1-kube-api-access-72v9b\") pod \"kas-connection-checker-685895c7c9-dnb6d\" (UID: \"30b966c5-d3d0-4743-a041-fe5f0f196fd1\") " pod="kube-system/kas-connection-checker-685895c7c9-dnb6d"
Nov 03 20:39:46.939615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939596    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 03 20:39:46.939821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939799    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47h9w\" (UniqueName: \"kubernetes.io/projected/90155d8a-eff2-4b22-ac1b-975bffe1d644-kube-api-access-47h9w\") pod \"kas-connection-checker-685895c7c9-l9wgt\" (UID: \"90155d8a-eff2-4b22-ac1b-975bffe1d644\") " pod="kube-system/kas-connection-checker-685895c7c9-l9wgt"
Nov 03 20:39:46.939981 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.939963    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwzrq\" (UniqueName: \"kubernetes.io/projected/f76fe3d1-1a83-4e2b-867c-792eba90309f-kube-api-access-mwzrq\") pod \"cluster-monitoring-operator-884576c57-lbb9n\" (UID: \"f76fe3d1-1a83-4e2b-867c-792eba90309f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-lbb9n"
Nov 03 20:39:46.940120 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940095    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96ml\" (UniqueName: \"kubernetes.io/projected/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-kube-api-access-r96ml\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.940183 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940124    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fns9\" (UniqueName: \"kubernetes.io/projected/bb233d06-f91f-4db9-be7b-bb9966b67a75-kube-api-access-6fns9\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.940183 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940151    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46q7\" (UniqueName: \"kubernetes.io/projected/476f59f4-3347-4e66-b0ea-51add44cfc27-kube-api-access-f46q7\") pod \"network-check-target-sl28h\" (UID: \"476f59f4-3347-4e66-b0ea-51add44cfc27\") " pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:46.940321 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940300    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp5qz\" (UniqueName: \"kubernetes.io/projected/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-kube-api-access-kp5qz\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.940390 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940357    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djxhv\" (UniqueName: \"kubernetes.io/projected/9e62edc4-aa68-49ad-ae64-909e88517f0f-kube-api-access-djxhv\") pod \"service-ca-5fffc97bdf-r2rmg\" (UID: \"9e62edc4-aa68-49ad-ae64-909e88517f0f\") " pod="openshift-service-ca/service-ca-5fffc97bdf-r2rmg"
Nov 03 20:39:46.940623 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940605    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqjr\" (UniqueName: \"kubernetes.io/projected/54725514-9087-42f7-8c47-5a995d71a0ee-kube-api-access-8cqjr\") pod \"network-check-source-6f78d59f99-f6gzh\" (UID: \"54725514-9087-42f7-8c47-5a995d71a0ee\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-f6gzh"
Nov 03 20:39:46.940775 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940752    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlhzq\" (UniqueName: \"kubernetes.io/projected/2e5e0dcf-8842-4b4a-bc4b-d11e653865fb-kube-api-access-vlhzq\") pod \"kas-connection-checker-685895c7c9-w4kw4\" (UID: \"2e5e0dcf-8842-4b4a-bc4b-d11e653865fb\") " pod="kube-system/kas-connection-checker-685895c7c9-w4kw4"
Nov 03 20:39:46.940846 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940771    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj8wv\" (UniqueName: \"kubernetes.io/projected/8e590e6f-ebec-4c39-8f00-d9460e3aeb57-kube-api-access-kj8wv\") pod \"migrator-66df5d496b-wjr2w\" (UID: \"8e590e6f-ebec-4c39-8f00-d9460e3aeb57\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wjr2w"
Nov 03 20:39:46.940910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940849    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8qts\" (UniqueName: \"kubernetes.io/projected/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-kube-api-access-s8qts\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.940910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940869    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.940910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.940902    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kx9d\" (UniqueName: \"kubernetes.io/projected/84298777-7d2a-4d90-80aa-0cd0c65e04f3-kube-api-access-2kx9d\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.941092 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941074    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvc7\" (UniqueName: \"kubernetes.io/projected/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-kube-api-access-hwvc7\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.941218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941197    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.941291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941239    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xpb\" (UniqueName: \"kubernetes.io/projected/83806e59-a55b-4491-b6d0-59b373bac613-kube-api-access-76xpb\") pod \"service-ca-operator-544d749c8c-j4jph\" (UID: \"83806e59-a55b-4491-b6d0-59b373bac613\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph"
Nov 03 20:39:46.941372 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941351    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-979r2\" (UniqueName: \"kubernetes.io/projected/b768b17b-8769-4002-8c89-3bee5fda2060-kube-api-access-979r2\") pod \"network-metrics-daemon-ffxrp\" (UID: \"b768b17b-8769-4002-8c89-3bee5fda2060\") " pod="openshift-multus/network-metrics-daemon-ffxrp"
Nov 03 20:39:46.941435 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941357    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqr7\" (UniqueName: \"kubernetes.io/projected/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-kube-api-access-zwqr7\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.941490 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941456    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cbl\" (UniqueName: \"kubernetes.io/projected/60bfa38c-92bc-4bef-ac05-18410621ea88-kube-api-access-v9cbl\") pod \"dns-default-zjglm\" (UID: \"60bfa38c-92bc-4bef-ac05-18410621ea88\") " pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:46.941744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941722    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cr8r\" (UniqueName: \"kubernetes.io/projected/4c2480e6-eaa3-402a-b3fc-2c63f26ff172-kube-api-access-4cr8r\") pod \"router-default-5769c487dc-z9pqs\" (UID: \"4c2480e6-eaa3-402a-b3fc-2c63f26ff172\") " pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:46.941823 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941787    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.941875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941862    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssmxp\" (UniqueName: \"kubernetes.io/projected/78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe-kube-api-access-ssmxp\") pod \"migrator-66df5d496b-9zp4f\" (UID: \"78df5c50-3dd5-4a0e-a01e-8f790b7dd1fe\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9zp4f"
Nov 03 20:39:46.941875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941869    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4j8h\" (UniqueName: \"kubernetes.io/projected/c6d5af76-ea42-41d7-9920-db291e351c53-kube-api-access-s4j8h\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.941980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941958    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.942034 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.941962    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrs8\" (UniqueName: \"kubernetes.io/projected/6c893222-ab83-4a89-98e2-169613906e67-kube-api-access-xvrs8\") pod \"volume-data-source-validator-58c9f5f46d-v6ssz\" (UID: \"6c893222-ab83-4a89-98e2-169613906e67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-v6ssz"
Nov 03 20:39:46.942167 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.942150    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjdn\" (UniqueName: \"kubernetes.io/projected/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-kube-api-access-bbjdn\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.942316 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.942296    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.942408 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.942371    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ggk7\" (UniqueName: \"kubernetes.io/projected/91453c10-6f59-4fd5-bf2a-69c2c3944375-kube-api-access-2ggk7\") pod \"ingress-canary-6sv74\" (UID: \"91453c10-6f59-4fd5-bf2a-69c2c3944375\") " pod="openshift-ingress-canary/ingress-canary-6sv74"
Nov 03 20:39:46.942464 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.942383    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9s5\" (UniqueName: \"kubernetes.io/projected/657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d-kube-api-access-gf9s5\") pod \"cluster-samples-operator-6ddb6c8b94-qmlbq\" (UID: \"657b6f32-d5b4-48c5-96ad-03cdfbfc7c8d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-qmlbq"
Nov 03 20:39:46.942520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.942492    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxkq8\" (UniqueName: \"kubernetes.io/projected/1c22090d-43fa-4fa7-adac-ecff9757f988-kube-api-access-hxkq8\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.943195 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.943174    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"image-registry-98474cc5c-vc78r\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") " pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:46.943266 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.943233    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c22090d-43fa-4fa7-adac-ecff9757f988-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-zrjt5\" (UID: \"1c22090d-43fa-4fa7-adac-ecff9757f988\") " pod="openshift-insights/insights-operator-7c4986d449-zrjt5"
Nov 03 20:39:46.943477 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.943457    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"periodic-gathering-lbpvw-klqsm\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") " pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:39:46.946501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.946479    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d37aa0c-7893-407a-a52a-46d8a1ffd00e-trusted-ca\") pod \"console-operator-7cd56b8dbc-xv2zb\" (UID: \"6d37aa0c-7893-407a-a52a-46d8a1ffd00e\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:46.948047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.948026    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"image-registry-5cf6c99f88-pj2bd\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") " pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:39:46.950853 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950837    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.950898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950861    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.950898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950877    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.950898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950892    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950906    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950923    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950946    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950954    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysconfig\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950972    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950978    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-device-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950997    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950997    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-run\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.950996    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951003    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-sys-fs\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951026    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-k8s-cni-cncf-io\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951040    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951031    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951066    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951073    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-etc-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951081    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-systemd\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951103    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-sysctl-conf\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951115    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951134    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-log-socket\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951156    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66a0afd2-dcb3-48ab-905d-4e4abab2fe97-host-slash\") pod \"iptables-alerter-c8m6k\" (UID: \"66a0afd2-dcb3-48ab-905d-4e4abab2fe97\") " pod="openshift-network-operator/iptables-alerter-c8m6k"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951151    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951200    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97-crio-socket\") pod \"insights-runtime-extractor-tph4g\" (UID: \"b8ce0b31-7cb3-47f5-82a5-7ea11cfc5a97\") " pod="openshift-insights/insights-runtime-extractor-tph4g"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951204    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951234    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951240    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-bin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951264    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951281    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-conf-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951288    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951315    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951340    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-kubelet-config\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951318    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-var-lib-kubelet\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951340    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951374    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-slash\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951397    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951417    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951436    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951446    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-modprobe-d\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951466    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951481    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-cni-multus\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951491    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951495    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-multus-certs\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951469    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-etc-kubernetes\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951506    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-etc-kubernetes\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951522    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951569    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84298777-7d2a-4d90-80aa-0cd0c65e04f3-hosts-file\") pod \"node-resolver-66w89\" (UID: \"84298777-7d2a-4d90-80aa-0cd0c65e04f3\") " pod="openshift-dns/node-resolver-66w89"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951572    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951599    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951609    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-lib-modules\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951626    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951650    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951657    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-sys\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951662    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-netns\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951674    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951679    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.951660 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951685    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-netd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951697    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951720    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951758    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-var-lib-openvswitch\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951762    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-etc-selinux\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951757    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951782    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-var-lib-kubelet\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951800    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951776    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-ovn\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951816    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951831    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951848    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951856    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951888    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-cnibin\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951886    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-registration-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951880    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951915    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951920    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-cnibin\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951924    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-run-ovn-kubernetes\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951949    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-cni-bin\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951955    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951979    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.951984    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c6d5af76-ea42-41d7-9920-db291e351c53-system-cni-dir\") pod \"multus-additional-cni-plugins-vc9ll\" (UID: \"c6d5af76-ea42-41d7-9920-db291e351c53\") " pod="openshift-multus/multus-additional-cni-plugins-vc9ll"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952009    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952017    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952031    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952046    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952065    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952097    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952100    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-socket-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952109    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51834fed-df6e-444b-9c2d-4501e262df12-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nx8kj\" (UID: \"51834fed-df6e-444b-9c2d-4501e262df12\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nx8kj"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952101    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-node-log\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952119    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952129    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-host-kubelet\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952131    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-host-run-netns\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952151    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/87fef4f1-d40d-4a12-9e0f-371838f9ecfa-host\") pod \"node-ca-4cjfz\" (UID: \"87fef4f1-d40d-4a12-9e0f-371838f9ecfa\") " pod="openshift-image-registry/node-ca-4cjfz"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952146    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952176    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952190    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952203    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952220    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952234    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952241    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-run-systemd\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952241    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-system-cni-dir\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952253    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-hostroot\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952254    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952276    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4f8781a-81f9-43f9-bd78-c5bcf5191fc0-multus-socket-dir-parent\") pod \"multus-b8zc5\" (UID: \"b4f8781a-81f9-43f9-bd78-c5bcf5191fc0\") " pod="openshift-multus/multus-b8zc5"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952288    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5084c67b-30c3-4ee8-a72c-a2f76b1c8c64-systemd-units\") pod \"ovnkube-node-gft95\" (UID: \"5084c67b-30c3-4ee8-a72c-a2f76b1c8c64\") " pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952291    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5713f5a0-c8bf-44c1-aeff-c9c580746c45-dbus\") pod \"global-pull-secret-syncer-xjbt9\" (UID: \"5713f5a0-c8bf-44c1-aeff-c9c580746c45\") " pod="kube-system/global-pull-secret-syncer-xjbt9"
Nov 03 20:39:46.952845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:46.952290    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb233d06-f91f-4db9-be7b-bb9966b67a75-host\") pod \"tuned-62lhn\" (UID: \"bb233d06-f91f-4db9-be7b-bb9966b67a75\") " pod="openshift-cluster-node-tuning-operator/tuned-62lhn"
Nov 03 20:39:47.034941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.034916    5915 scope.go:122] "RemoveContainer" containerID="3cbbacbf0aa85e786b363ec95dc3b1010030f98b85dd53ebcf0de2c437382895"
Nov 03 20:39:47.187149 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.187082    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:47.756498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.756464    5915 generic.go:356] "Generic (PLEG): container finished" podID="1c22090d-43fa-4fa7-adac-ecff9757f988" containerID="9d5effa72a0949b1aacdedb1d8d3d75f7ec0c1c9370f21fd0c0e13793739cc06" exitCode=0
Nov 03 20:39:47.756944 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.756537    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zrjt5" event={"ID":"1c22090d-43fa-4fa7-adac-ecff9757f988","Type":"ContainerDied","Data":"9d5effa72a0949b1aacdedb1d8d3d75f7ec0c1c9370f21fd0c0e13793739cc06"}
Nov 03 20:39:47.756944 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.756861    5915 scope.go:122] "RemoveContainer" containerID="9d5effa72a0949b1aacdedb1d8d3d75f7ec0c1c9370f21fd0c0e13793739cc06"
Nov 03 20:39:47.758220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.758199    5915 generic.go:356] "Generic (PLEG): container finished" podID="bab0970d-6322-42d0-a66b-6d0acba24275" containerID="5ae1bcca9392072495d748bf0bea39324a2809632ed3bf1b41f19c7f1777f95f" exitCode=0
Nov 03 20:39:47.758314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.758269    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx" event={"ID":"bab0970d-6322-42d0-a66b-6d0acba24275","Type":"ContainerDied","Data":"5ae1bcca9392072495d748bf0bea39324a2809632ed3bf1b41f19c7f1777f95f"}
Nov 03 20:39:47.758548 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.758532    5915 scope.go:122] "RemoveContainer" containerID="5ae1bcca9392072495d748bf0bea39324a2809632ed3bf1b41f19c7f1777f95f"
Nov 03 20:39:47.759873 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.759852    5915 generic.go:356] "Generic (PLEG): container finished" podID="83806e59-a55b-4491-b6d0-59b373bac613" containerID="9eda50cde32c01dbef48fdaa9377dbe4f560e528f9c6345e47b3d427c6540f92" exitCode=0
Nov 03 20:39:47.759958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.759924    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph" event={"ID":"83806e59-a55b-4491-b6d0-59b373bac613","Type":"ContainerDied","Data":"9eda50cde32c01dbef48fdaa9377dbe4f560e528f9c6345e47b3d427c6540f92"}
Nov 03 20:39:47.760182 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.760166    5915 scope.go:122] "RemoveContainer" containerID="9eda50cde32c01dbef48fdaa9377dbe4f560e528f9c6345e47b3d427c6540f92"
Nov 03 20:39:47.762156 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.762139    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:39:47.762303 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.762282    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb" event={"ID":"6d37aa0c-7893-407a-a52a-46d8a1ffd00e","Type":"ContainerStarted","Data":"02c7ef9a1dc29d98830b7c9bb99b54cdb5ff81ac468772188b04c53726c796e0"}
Nov 03 20:39:47.762588 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.762548    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:47.903709 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:47.902936    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:48.039717 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.039646    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:48.047500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.047478    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sl28h"
Nov 03 20:39:48.073348 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.073326    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-xv2zb"
Nov 03 20:39:48.334296 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.334221    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5769c487dc-z9pqs"
Nov 03 20:39:48.372832 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.372785    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:48.619216 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.619124    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:48.768340 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.768301    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-k2jpx" event={"ID":"bab0970d-6322-42d0-a66b-6d0acba24275","Type":"ContainerStarted","Data":"f68008ce5e2e3d53cc915bb8a2988d0dc41b7f5d51a16999aefcccafa1caaa18"}
Nov 03 20:39:48.770258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.770233    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-j4jph" event={"ID":"83806e59-a55b-4491-b6d0-59b373bac613","Type":"ContainerStarted","Data":"968bcdd860f0e31c896532975c0d088b9ae906be54f7555584918d8f9a5601a4"}
Nov 03 20:39:48.772485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.772461    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-zrjt5" event={"ID":"1c22090d-43fa-4fa7-adac-ecff9757f988","Type":"ContainerStarted","Data":"367c88f6761dc0c9290a997be6b651f2931201d522f60e322ce7e5564051fcf8"}
Nov 03 20:39:48.813245 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:48.813200    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:49.057031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:49.056998    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:49.235231 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:49.235199    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:49.543108 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:49.543078    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:49.811092 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:49.811022    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:49.984154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:49.984119    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:50.905632 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:50.905604    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zjglm"
Nov 03 20:39:51.234024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:51.234000    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:39:51.234228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:51.234216    5915 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 20:39:53.214284 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:53.214251    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:53.359699 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:53.359665    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:53.717614 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:53.717586    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:53.863963 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:53.863930    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:54.261717 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:54.261690    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:54.515216 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:54.515145    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:54.697112 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:54.697080    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:54.915141 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:54.915067    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:55.086074 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.086048    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:55.234877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.234846    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rqrcb"]
Nov 03 20:39:55.237170 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.237155    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.301187 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.301161    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:55.313126 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.313099    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 20:39:55.320394 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320373    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-tls\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320402    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320420    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-sys\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320448    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-textfile\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320497    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320646 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320612    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-metrics-client-ca\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320646 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320632    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfcfv\" (UniqueName: \"kubernetes.io/projected/2baeac45-94a5-4d6b-85ea-a257c2f6482f-kube-api-access-jfcfv\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320802 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320672    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-wtmp\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.320802 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.320691    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-root\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.343896 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.343874    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 20:39:55.343992 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.343948    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 20:39:55.344035 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.344027    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 20:39:55.344071 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.344062    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzwv7\""
Nov 03 20:39:55.421368 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421336    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-wtmp\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421368 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421370    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-root\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421392    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-tls\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421419    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421443    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-sys\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421454    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-root\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421476    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-textfile\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421497    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421524    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-metrics-client-ca\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421521    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-wtmp\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421521    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2baeac45-94a5-4d6b-85ea-a257c2f6482f-sys\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.421821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421547    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfcfv\" (UniqueName: \"kubernetes.io/projected/2baeac45-94a5-4d6b-85ea-a257c2f6482f-kube-api-access-jfcfv\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.422013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421871    5915 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 20:39:55.422013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.421949    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-textfile\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.422150 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.422114    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.422258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.422218    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2baeac45-94a5-4d6b-85ea-a257c2f6482f-metrics-client-ca\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.424030 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.424014    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-tls\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.424135 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.424115    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2baeac45-94a5-4d6b-85ea-a257c2f6482f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.482526 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.482498    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfcfv\" (UniqueName: \"kubernetes.io/projected/2baeac45-94a5-4d6b-85ea-a257c2f6482f-kube-api-access-jfcfv\") pod \"node-exporter-rqrcb\" (UID: \"2baeac45-94a5-4d6b-85ea-a257c2f6482f\") " pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.521362 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.521301    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:55.556345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.556320    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rqrcb"
Nov 03 20:39:55.567707 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:39:55.567679    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2baeac45_94a5_4d6b_85ea_a257c2f6482f.slice/crio-b7f6ee9ef0c356aaf7381226fc2f2bb6486824aeb3be6dd28fc50dbab5f9497f WatchSource:0}: Error finding container b7f6ee9ef0c356aaf7381226fc2f2bb6486824aeb3be6dd28fc50dbab5f9497f: Status 404 returned error can't find the container with id b7f6ee9ef0c356aaf7381226fc2f2bb6486824aeb3be6dd28fc50dbab5f9497f
Nov 03 20:39:55.569479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.569465    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:39:55.737691 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.737665    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:39:55.768027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.767995    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:55.802127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:55.802065    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rqrcb" event={"ID":"2baeac45-94a5-4d6b-85ea-a257c2f6482f","Type":"ContainerStarted","Data":"b7f6ee9ef0c356aaf7381226fc2f2bb6486824aeb3be6dd28fc50dbab5f9497f"}
Nov 03 20:39:56.189014 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:56.188935    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:56.351211 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:56.351179    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:56.505642 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:56.505609    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:56.754261 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:56.754220    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:57.362917 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:57.362846    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:57.812213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:57.812179    5915 generic.go:356] "Generic (PLEG): container finished" podID="2baeac45-94a5-4d6b-85ea-a257c2f6482f" containerID="7f98a759e4743f550b206c11b9d5a4211c39c3e9ca18432ae8ebe94c881dd28e" exitCode=0
Nov 03 20:39:57.812399 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:57.812265    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rqrcb" event={"ID":"2baeac45-94a5-4d6b-85ea-a257c2f6482f","Type":"ContainerDied","Data":"7f98a759e4743f550b206c11b9d5a4211c39c3e9ca18432ae8ebe94c881dd28e"}
Nov 03 20:39:58.821495 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:58.821463    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rqrcb" event={"ID":"2baeac45-94a5-4d6b-85ea-a257c2f6482f","Type":"ContainerStarted","Data":"f8fbbcde15e3d55159bc9dbbec26d821e28bf58c3b74f5f85eb593dc8198067c"}
Nov 03 20:39:58.821895 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:58.821540    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rqrcb" event={"ID":"2baeac45-94a5-4d6b-85ea-a257c2f6482f","Type":"ContainerStarted","Data":"f22ee55db2389245a4700bb50cc15651bd9930808cfcc248d0d3ff5ad0e797d7"}
Nov 03 20:39:58.989878 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:58.989823    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rqrcb" podStartSLOduration=2.394792434 podStartE2EDuration="3.989809193s" totalImagesPullingTime="1.518722435s" totalInitContainerRuntime="76.294324ms" isStatefulPod=true podCreationTimestamp="2026-11-03 20:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:58.98887236 +0000 UTC m=+13.440241538" watchObservedRunningTime="2026-11-03 20:39:58.989809193 +0000 UTC m=+13.441178427"
Nov 03 20:39:59.009632 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:59.009611    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cks2d"
Nov 03 20:39:59.354184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:59.354150    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:59.520894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:59.520860    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:59.673616 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:59.673523    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:39:59.839484 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:39:59.839452    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:00.001763 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:00.001737    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:01.151828 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:01.151802    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:40:01.262697 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:01.262668    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:01.409088 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:01.409010    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:01.554537 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:01.554509    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:01.698635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:01.698547    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:03.242874 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:03.242845    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:03.445541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:03.445511    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:04.254614 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:04.254586    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:04.680274 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:04.680193    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:04.882089 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:04.882057    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:05.025201 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:05.025175    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:05.467450 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:05.467416    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:05.615729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:05.615701    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:06.423891 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:06.423855    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:06.626135 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:06.626105    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:07.002226 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:40:07.002194    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:40:07.176209 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:07.176181    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:07.324890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:07.324822    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:07.967374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:07.967334    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:08.109768 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:08.109737    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:11.083096 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.083063    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:40:11.083604 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.083276    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" podUID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" containerName="insights-gathering" containerID="cri-o://5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512" gracePeriod=30
Nov 03 20:40:11.388882 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.388848    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:40:11.405077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.405044    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:11.454287 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454257    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") pod \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") "
Nov 03 20:40:11.454437 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454323    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") pod \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") "
Nov 03 20:40:11.454437 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454353    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") pod \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") "
Nov 03 20:40:11.454437 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454403    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") pod \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\" (UID: \"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32\") "
Nov 03 20:40:11.454678 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454650    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path" pod "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" (UID: "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:40:11.454746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454729    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle" pod "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" (UID: "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:11.454800 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.454767    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle" pod "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" (UID: "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:11.456913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.456889    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb" pod "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" (UID: "9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32"). InnerVolumeSpecName "kube-api-access-jhxxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:11.555240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.555211    5915 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-archives-path\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:11.555240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.555237    5915 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-trusted-ca-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:11.555240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.555247    5915 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-service-ca-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:11.555436 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.555256    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhxxb\" (UniqueName: \"kubernetes.io/projected/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32-kube-api-access-jhxxb\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:11.565778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.565741    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:11.754618 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.754579    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerName="registry" containerID="cri-o://4704c02d9eb88544e953bbff27017713c05ea0469a9b67a25f726d7be26a388c" gracePeriod=30
Nov 03 20:40:11.875668 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.875641    5915 generic.go:356] "Generic (PLEG): container finished" podID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerID="4704c02d9eb88544e953bbff27017713c05ea0469a9b67a25f726d7be26a388c" exitCode=0
Nov 03 20:40:11.875814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.875711    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" event={"ID":"1747a6b1-0e38-4948-baa2-a75cbe265752","Type":"ContainerDied","Data":"4704c02d9eb88544e953bbff27017713c05ea0469a9b67a25f726d7be26a388c"}
Nov 03 20:40:11.876769 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.876744    5915 generic.go:356] "Generic (PLEG): container finished" podID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" containerID="5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512" exitCode=2
Nov 03 20:40:11.876884 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.876791    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lbpvw-klqsm"
Nov 03 20:40:11.876884 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.876817    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" event={"ID":"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32","Type":"ContainerDied","Data":"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"}
Nov 03 20:40:11.876884 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.876857    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lbpvw-klqsm" event={"ID":"9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32","Type":"ContainerDied","Data":"e15b1e950f36eed0ceae1a7bc75893bbb17c39addf5d588ddc7cdae700a19224"}
Nov 03 20:40:11.876884 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.876879    5915 scope.go:122] "RemoveContainer" containerID="5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"
Nov 03 20:40:11.884860 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.884842    5915 scope.go:122] "RemoveContainer" containerID="5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"
Nov 03 20:40:11.885104 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:40:11.885084    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512\": container with ID starting with 5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512 not found: ID does not exist" containerID="5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"
Nov 03 20:40:11.885163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:11.885113    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512"} err="failed to get container status \"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512\": rpc error: code = NotFound desc = could not find container \"5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512\": container with ID starting with 5a42e4d337c819b9e26e2a3f835027f2ede24b9dd1a11e0efb2cf150016b0512 not found: ID does not exist"
Nov 03 20:40:12.036919 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.036888    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:40:12.043123 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.043073    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:40:12.047611 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.047589    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-lbpvw-klqsm"]
Nov 03 20:40:12.159807 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159780    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159828    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159871    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159907    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159961    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.159991    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.160031    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.160066    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") pod \"1747a6b1-0e38-4948-baa2-a75cbe265752\" (UID: \"1747a6b1-0e38-4948-baa2-a75cbe265752\") "
Nov 03 20:40:12.160739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.160674    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:12.160739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.160722    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:12.162526 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.162502    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:40:12.162950 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.162914    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:40:12.163052 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.162994    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "kube-api-access-gbz78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:12.163124 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.163105    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:12.163213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.163188    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:12.169191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.169163    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted" pod "1747a6b1-0e38-4948-baa2-a75cbe265752" (UID: "1747a6b1-0e38-4948-baa2-a75cbe265752"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:40:12.261576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261533    5915 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1747a6b1-0e38-4948-baa2-a75cbe265752-ca-trust-extracted\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261577    5915 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-certificates\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261589    5915 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-registry-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261597    5915 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1747a6b1-0e38-4948-baa2-a75cbe265752-trusted-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261607    5915 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-bound-sa-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261616    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbz78\" (UniqueName: \"kubernetes.io/projected/1747a6b1-0e38-4948-baa2-a75cbe265752-kube-api-access-gbz78\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261624    5915 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-installation-pull-secrets\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.261739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.261633    5915 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1747a6b1-0e38-4948-baa2-a75cbe265752-image-registry-private-configuration\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:12.886025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.885987    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd" event={"ID":"1747a6b1-0e38-4948-baa2-a75cbe265752","Type":"ContainerDied","Data":"ab97a79061ca803703c79733ee99878793aa33d41be44fc409a89f67f9c75516"}
Nov 03 20:40:12.886218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.886070    5915 scope.go:122] "RemoveContainer" containerID="4704c02d9eb88544e953bbff27017713c05ea0469a9b67a25f726d7be26a388c"
Nov 03 20:40:12.886218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:12.886003    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf6c99f88-pj2bd"
Nov 03 20:40:13.098322 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:13.098283    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:40:13.104618 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:13.104593    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cf6c99f88-pj2bd"]
Nov 03 20:40:13.678323 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:13.678283    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" path="/var/lib/kubelet/pods/1747a6b1-0e38-4948-baa2-a75cbe265752/volumes"
Nov 03 20:40:13.678870 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:13.678849    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" path="/var/lib/kubelet/pods/9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32/volumes"
Nov 03 20:40:13.711128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:13.711106    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gft95"
Nov 03 20:40:21.401978 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:21.401933    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:21.544368 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:21.544330    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:23.064764 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:23.064728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:23.212381 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:23.212353    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:23.720027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:23.719996    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:23.867588 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:23.867540    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:26.171890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.171853    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry" containerID="cri-o://6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af" gracePeriod=30
Nov 03 20:40:26.418089 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.418066    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:40:26.578090 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578059    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578124    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578150    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578263    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578333    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578373    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578411    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578553 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578501    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") pod \"0f528bfd-d20d-4fe1-81ae-754987ca453f\" (UID: \"0f528bfd-d20d-4fe1-81ae-754987ca453f\") "
Nov 03 20:40:26.578932 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.578832    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:26.579129 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.579104    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:40:26.581044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.581013    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:40:26.581229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.581151    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:26.581299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.581245    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:26.581299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.581287    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:40:26.581381 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.581327    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "kube-api-access-zwn5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:40:26.587270 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.587240    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted" pod "0f528bfd-d20d-4fe1-81ae-754987ca453f" (UID: "0f528bfd-d20d-4fe1-81ae-754987ca453f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:40:26.679689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679662    5915 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-trusted-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679686    5915 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-bound-sa-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679696    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwn5d\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-kube-api-access-zwn5d\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679706    5915 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679715    5915 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0f528bfd-d20d-4fe1-81ae-754987ca453f-ca-trust-extracted\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679725    5915 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0f528bfd-d20d-4fe1-81ae-754987ca453f-registry-certificates\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679733    5915 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-installation-pull-secrets\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.679894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.679742    5915 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0f528bfd-d20d-4fe1-81ae-754987ca453f-image-registry-private-configuration\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:40:26.945232 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.945156    5915 generic.go:356] "Generic (PLEG): container finished" podID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerID="6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af" exitCode=0
Nov 03 20:40:26.945232 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.945223    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" event={"ID":"0f528bfd-d20d-4fe1-81ae-754987ca453f","Type":"ContainerDied","Data":"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"}
Nov 03 20:40:26.945232 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.945225    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-98474cc5c-vc78r"
Nov 03 20:40:26.945441 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.945284    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-98474cc5c-vc78r" event={"ID":"0f528bfd-d20d-4fe1-81ae-754987ca453f","Type":"ContainerDied","Data":"4728685f325537de71f12f84c407c2ab38cac8f77616032e0a16a8b8fc5f26e9"}
Nov 03 20:40:26.945441 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.945304    5915 scope.go:122] "RemoveContainer" containerID="6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"
Nov 03 20:40:26.957351 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.957335    5915 scope.go:122] "RemoveContainer" containerID="6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"
Nov 03 20:40:26.957627 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:40:26.957608    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af\": container with ID starting with 6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af not found: ID does not exist" containerID="6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"
Nov 03 20:40:26.957685 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:26.957636    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af"} err="failed to get container status \"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af\": rpc error: code = NotFound desc = could not find container \"6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af\": container with ID starting with 6fce6f87f174571403f44688b6718f0420182ca14bfad6bdcc389eb069e914af not found: ID does not exist"
Nov 03 20:40:27.108125 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:27.108096    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:40:27.109985 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:27.109960    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-98474cc5c-vc78r"]
Nov 03 20:40:27.678448 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:27.678415    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" path="/var/lib/kubelet/pods/0f528bfd-d20d-4fe1-81ae-754987ca453f/volumes"
Nov 03 20:40:31.213749 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:31.213713    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:31.359096 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:31.359064    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:32.617025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:32.616995    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:32.758189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:32.758159    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:37.002401 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:40:37.002367    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:40:38.713823 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:38.713789    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:38.859889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:38.859853    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:41.876130 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:41.876092    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:42.019044 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:42.019015    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:49.449239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:49.449200    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:49.592378 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:49.592348    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:50.712771 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:50.712732    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:50.860695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:50.860656    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:53.716504 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:53.716472    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:53.858926 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:53.858889    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:57.781313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:57.781277    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:40:57.927716 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:40:57.927674    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:06.796982 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:06.796938    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:07.002932 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:41:07.002895    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:41:08.725552 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:08.725515    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:08.868571 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:08.868513    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:37.000893 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:41:37.000806    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:41:47.844843 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:47.844805    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:51.334972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:51.334942    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:51.478861 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:51.478829    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:41:55.068508 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:41:55.068472    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:42:06.216118 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:42:06.216087    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:42:07.001948 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:42:07.001914    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:42:07.069013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:42:07.068979    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:42:07.212048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:42:07.212015    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:42:37.001662 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:42:37.001625    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:43:07.001795 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:43:07.001722    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:43:07.483438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:07.483405    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:17.353741 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:17.353709    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:17.554496 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:17.554462    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:18.967280 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:18.967247    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:19.129669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:19.129637    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:20.079941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.079910    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"]
Nov 03 20:43:20.080421 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080397    5915 cpu_manager.go:414] "containerMap: removing container" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry"
Nov 03 20:43:20.080421 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080419    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080444    5915 cpu_manager.go:414] "containerMap: removing container" podUID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" containerName="insights-gathering"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080452    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" containerName="insights-gathering"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080463    5915 cpu_manager.go:414] "containerMap: removing container" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerName="registry"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080469    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerName="registry"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080601    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f528bfd-d20d-4fe1-81ae-754987ca453f" containerName="registry"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080620    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d1f9441-f48b-4ba2-bfc3-e48b69ec5d32" containerName="insights-gathering"
Nov 03 20:43:20.080659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.080629    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="1747a6b1-0e38-4948-baa2-a75cbe265752" containerName="registry"
Nov 03 20:43:20.083834 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.083812    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.185530 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.185502    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 20:43:20.185530 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.185505    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 20:43:20.185760 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.185530    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-sqvsr\""
Nov 03 20:43:20.186408 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.186391    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"]
Nov 03 20:43:20.186865 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.186847    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"]
Nov 03 20:43:20.190023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.190009    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.235784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.235761    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"]
Nov 03 20:43:20.238930 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.238916    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.242607 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.242591    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-br9qr\""
Nov 03 20:43:20.246700 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.246682    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.246792 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.246708    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrpkw\" (UniqueName: \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.246839 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.246807    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.258627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.258608    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"]
Nov 03 20:43:20.262136 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.262087    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"]
Nov 03 20:43:20.265452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.265432    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.290650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.290624    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-bdsdc\""
Nov 03 20:43:20.323063 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.323039    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"]
Nov 03 20:43:20.327942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.327913    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"]
Nov 03 20:43:20.337345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.337284    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-kwwj4\""
Nov 03 20:43:20.341235 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.341213    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:20.347214 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347193    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.347304 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347229    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.347304 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347263    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.347377 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347335    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc5pz\" (UniqueName: \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.347419 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.347467 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347445    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrpkw\" (UniqueName: \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.347511 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347489    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8fnw\" (UniqueName: \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.347581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347528    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.347581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347531    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.347740 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347618    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.347740 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.347715    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.368873 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.368848    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"]
Nov 03 20:43:20.372107 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.372092    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.408508 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.408480    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrpkw\" (UniqueName: \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.424129 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.424107    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-dmcg5\""
Nov 03 20:43:20.432033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.432015    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"]
Nov 03 20:43:20.448415 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448391    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.448498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448422    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.448498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448448    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.448596 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448505    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.448596 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448551    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.448661 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448612    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.448754 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448733    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc5pz\" (UniqueName: \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.448844 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448827    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8fnw\" (UniqueName: \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.448902 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448870    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmmml\" (UniqueName: \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.448954 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448896    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.448954 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448920    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.449033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.448959    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.449033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.449021    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.510570 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.510535    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc5pz\" (UniqueName: \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.510781 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.510756    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8fnw\" (UniqueName: \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550086    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmmml\" (UniqueName: \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550143    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550190    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550223    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550258    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68hrv\" (UniqueName: \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550312    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.550725    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.554542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.551308    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.554918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.554754    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:43:20.615261 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.612912    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmmml\" (UniqueName: \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.651349 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.651322    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.651493 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.651362    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.651493 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.651382    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68hrv\" (UniqueName: \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.651820 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.651784    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.651873 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.651792    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.698911 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.698886    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:43:20.714026 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.714006    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68hrv\" (UniqueName: \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.805831 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.805801    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:43:20.880889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.880856    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:43:20.926299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.926270    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"]
Nov 03 20:43:20.931513 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:20.931476    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda687e327_f01a_457b_ab4e_7be9f827d493.slice/crio-60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f WatchSource:0}: Error finding container 60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f: Status 404 returned error can't find the container with id 60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f
Nov 03 20:43:20.986771 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.986746    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:43:20.987775 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.987724    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"]
Nov 03 20:43:20.991219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:20.991201    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.044152 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.044104    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-ch5dz\""
Nov 03 20:43:21.049987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.049964    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"]
Nov 03 20:43:21.054965 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.054927    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q8f5\" (UniqueName: \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.055069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.054988    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.055163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.055140    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.068469 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.068441    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"]
Nov 03 20:43:21.071328 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:21.071303    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba3612bf_03b4_4a45_a621_77bde7cb7abf.slice/crio-0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796 WatchSource:0}: Error finding container 0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796: Status 404 returned error can't find the container with id 0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796
Nov 03 20:43:21.156244 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.156176    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.156244 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.156223    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q8f5\" (UniqueName: \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.156722 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.156267    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.156722 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.156540    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.156722 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.156653    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.175057 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.175030    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"]
Nov 03 20:43:21.178024 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:21.178001    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5014d06_637a_4e17_b7d1_4790aebbdc36.slice/crio-17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83 WatchSource:0}: Error finding container 17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83: Status 404 returned error can't find the container with id 17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83
Nov 03 20:43:21.216908 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.216884    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q8f5\" (UniqueName: \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.249860 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.249829    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"]
Nov 03 20:43:21.253756 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:21.253731    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efdc9bf_dcc5_403d_8cf7_2cf370cab9a5.slice/crio-b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b WatchSource:0}: Error finding container b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b: Status 404 returned error can't find the container with id b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b
Nov 03 20:43:21.308290 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.308253    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:43:21.358119 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.358077    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"]
Nov 03 20:43:21.363375 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:21.363340    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ffa83a_3863_4f0b_aa02_2988bf877e98.slice/crio-8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0 WatchSource:0}: Error finding container 8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0: Status 404 returned error can't find the container with id 8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0
Nov 03 20:43:21.572621 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.572588    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl" event={"ID":"55ffa83a-3863-4f0b-aa02-2988bf877e98","Type":"ContainerStarted","Data":"8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0"}
Nov 03 20:43:21.573774 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.573746    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz" event={"ID":"a687e327-f01a-457b-ab4e-7be9f827d493","Type":"ContainerStarted","Data":"60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f"}
Nov 03 20:43:21.574894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.574870    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982" event={"ID":"ba3612bf-03b4-4a45-a621-77bde7cb7abf","Type":"ContainerStarted","Data":"0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796"}
Nov 03 20:43:21.576058 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.576038    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerStarted","Data":"17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83"}
Nov 03 20:43:21.577060 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.577039    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerStarted","Data":"b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b"}
Nov 03 20:43:21.679032 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:21.679003    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"]
Nov 03 20:43:21.682226 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:21.682199    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804f327c_7502_47da_9bff_03021ab05656.slice/crio-3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae WatchSource:0}: Error finding container 3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae: Status 404 returned error can't find the container with id 3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae
Nov 03 20:43:22.586141 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:22.586041    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9" event={"ID":"804f327c-7502-47da-9bff-03021ab05656","Type":"ContainerStarted","Data":"3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae"}
Nov 03 20:43:23.960869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:23.960829    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"]
Nov 03 20:43:23.966462 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:23.966040    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:23.983751 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:23.983711    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:23.983898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:23.983788    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:23.983898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:23.983817    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27479\" (UniqueName: \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.070203 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.070165    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-dl28n\""
Nov 03 20:43:24.071303 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.071281    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"]
Nov 03 20:43:24.084501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.084478    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.084665 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.084526    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.084665 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.084548    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27479\" (UniqueName: \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.085438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.085369    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.085820 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.085773    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.148536 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.148473    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27479\" (UniqueName: \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.289936 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.289581    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:43:24.705640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:24.705551    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"]
Nov 03 20:43:24.715542 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:24.715507    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7387d9ec_bfa6_455d_99f8_ac98e19daf3a.slice/crio-f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5 WatchSource:0}: Error finding container f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5: Status 404 returned error can't find the container with id f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5
Nov 03 20:43:25.607823 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:25.607745    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerStarted","Data":"f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5"}
Nov 03 20:43:26.743333 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.743298    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"]
Nov 03 20:43:26.747541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.747519    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.778873 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.778841    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"]
Nov 03 20:43:26.783827 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.782791    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.810890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.810848    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.811042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.810930    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.811042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.811004    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64zr2\" (UniqueName: \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.811159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.811085    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk2nz\" (UniqueName: \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.811465 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.811187    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.811573 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.811513    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.828771 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.828742    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"]
Nov 03 20:43:26.830168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.830150    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"]
Nov 03 20:43:26.835676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.835659    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:26.851277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.851229    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"]
Nov 03 20:43:26.895674 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.893986    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"]
Nov 03 20:43:26.897875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.897848    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:26.901581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.901537    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"]
Nov 03 20:43:26.912578 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.912015    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:26.912578 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.912066    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.912578 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.912116    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.912578 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.912161    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-856tg\" (UniqueName: \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:26.913262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913239    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.913471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913452    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.913580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913542    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.913654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913623    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.913712 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913678    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64zr2\" (UniqueName: \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.913765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913713    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:26.913812 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913778    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk2nz\" (UniqueName: \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.913967 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.913928    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.914444 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.914299    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:26.981108 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.980114    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"]
Nov 03 20:43:26.983815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.983757    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk2nz\" (UniqueName: \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:26.989168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:26.989120    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64zr2\" (UniqueName: \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:27.012210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.012104    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:27.015051 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015023    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.015168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015082    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbrjh\" (UniqueName: \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.015168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015119    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.015397 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015361    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-856tg\" (UniqueName: \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.015483 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015405    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.015483 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015467    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.016128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015939    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.016128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.015996    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.069583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.069535    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:43:27.083242 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.083184    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-856tg\" (UniqueName: \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.105468 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.104962    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:43:27.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.116222    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.116290    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.116342    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbrjh\" (UniqueName: \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.117096    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.117340    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.175017 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.173791    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:43:27.198711 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.198681    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbrjh\" (UniqueName: \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.210498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.208466    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:27.226058 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.225829    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:43:27.537301 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.537273    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"]
Nov 03 20:43:27.542692 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:27.540194    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2233017_06ef_4c6e_bece_94c4128c467d.slice/crio-19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10 WatchSource:0}: Error finding container 19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10: Status 404 returned error can't find the container with id 19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10
Nov 03 20:43:27.566193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.566153    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"]
Nov 03 20:43:27.620438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.620383    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"]
Nov 03 20:43:27.628996 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.628910    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerStarted","Data":"79b4162ec3b329b13a679ac522d11fc1f3fe4b8aea0f9282f200069b743b0cfd"}
Nov 03 20:43:27.632371 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.632204    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw" event={"ID":"e2233017-06ef-4c6e-bece-94c4128c467d","Type":"ContainerStarted","Data":"19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10"}
Nov 03 20:43:27.661857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:27.661793    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"]
Nov 03 20:43:27.670190 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:43:27.670142    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdda7ae6_846d_4865_9040_8042e6348844.slice/crio-7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654 WatchSource:0}: Error finding container 7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654: Status 404 returned error can't find the container with id 7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654
Nov 03 20:43:28.517366 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:28.517300    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:28.667488 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:28.667405    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz" event={"ID":"bdda7ae6-846d-4865-9040-8042e6348844","Type":"ContainerStarted","Data":"7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654"}
Nov 03 20:43:28.677288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:28.677245    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x" event={"ID":"992a347e-1791-4721-8260-9665ec97e3bd","Type":"ContainerStarted","Data":"db3e053f1c07ba76bfbb010ed417a9f3e8941ee65a117ebb9f1ecc625d01092f"}
Nov 03 20:43:28.837966 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:28.837878    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:28.989974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:28.989910    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:32.472735 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:32.472633    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:37.141089 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:43:37.138962    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:43:40.803088 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.803013    5915 generic.go:356] "Generic (PLEG): container finished" podID="a687e327-f01a-457b-ab4e-7be9f827d493" containerID="cc8c567396d0501caeb101d75353663b5f1f90835d9632941911dbe18ebdac12" exitCode=0
Nov 03 20:43:40.803634 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.803100    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz" event={"ID":"a687e327-f01a-457b-ab4e-7be9f827d493","Type":"ContainerDied","Data":"cc8c567396d0501caeb101d75353663b5f1f90835d9632941911dbe18ebdac12"}
Nov 03 20:43:40.804747 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.804720    5915 generic.go:356] "Generic (PLEG): container finished" podID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerID="482a649dd80edfc88a5cf97dc586727619fa29212732069cc0b844baa3d37ef3" exitCode=0
Nov 03 20:43:40.804856 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.804813    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982" event={"ID":"ba3612bf-03b4-4a45-a621-77bde7cb7abf","Type":"ContainerDied","Data":"482a649dd80edfc88a5cf97dc586727619fa29212732069cc0b844baa3d37ef3"}
Nov 03 20:43:40.807132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.807104    5915 generic.go:356] "Generic (PLEG): container finished" podID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerID="81e06c086739db683e73ae21b637fa3cf946425efbf25d454474cefde4a47253" exitCode=0
Nov 03 20:43:40.807228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:40.807142    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerDied","Data":"81e06c086739db683e73ae21b637fa3cf946425efbf25d454474cefde4a47253"}
Nov 03 20:43:41.813240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.813206    5915 generic.go:356] "Generic (PLEG): container finished" podID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerID="2b6c8eb46e6eae69affa08cb8f6c8617440fde4c60e25b9885e74882ce1cba73" exitCode=0
Nov 03 20:43:41.813711 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.813289    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl" event={"ID":"55ffa83a-3863-4f0b-aa02-2988bf877e98","Type":"ContainerDied","Data":"2b6c8eb46e6eae69affa08cb8f6c8617440fde4c60e25b9885e74882ce1cba73"}
Nov 03 20:43:41.814956 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.814935    5915 generic.go:356] "Generic (PLEG): container finished" podID="e2233017-06ef-4c6e-bece-94c4128c467d" containerID="0408db6faafea3d006e598136d07ab69e83d09bff1995da1e6aa7a84deed7375" exitCode=0
Nov 03 20:43:41.815060 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.814995    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw" event={"ID":"e2233017-06ef-4c6e-bece-94c4128c467d","Type":"ContainerDied","Data":"0408db6faafea3d006e598136d07ab69e83d09bff1995da1e6aa7a84deed7375"}
Nov 03 20:43:41.816372 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.816352    5915 generic.go:356] "Generic (PLEG): container finished" podID="992a347e-1791-4721-8260-9665ec97e3bd" containerID="bb03dd85faf91f28209533d2ba85fd8a01f13c4124a2400627c0811a6072c0eb" exitCode=0
Nov 03 20:43:41.816453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:41.816426    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x" event={"ID":"992a347e-1791-4721-8260-9665ec97e3bd","Type":"ContainerDied","Data":"bb03dd85faf91f28209533d2ba85fd8a01f13c4124a2400627c0811a6072c0eb"}
Nov 03 20:43:42.823859 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.823827    5915 generic.go:356] "Generic (PLEG): container finished" podID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerID="83c3bc6463890a5e34ba94cfb811ba9ef431ed37ed7f01b36febae4c3ed1261e" exitCode=0
Nov 03 20:43:42.824273 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.823907    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982" event={"ID":"ba3612bf-03b4-4a45-a621-77bde7cb7abf","Type":"ContainerDied","Data":"83c3bc6463890a5e34ba94cfb811ba9ef431ed37ed7f01b36febae4c3ed1261e"}
Nov 03 20:43:42.825459 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.825439    5915 generic.go:356] "Generic (PLEG): container finished" podID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerID="a4c466eca8dfab2397ef8bc830b3a65bf204fde6267e5e20649afd701df3ae3b" exitCode=0
Nov 03 20:43:42.825584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.825532    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerDied","Data":"a4c466eca8dfab2397ef8bc830b3a65bf204fde6267e5e20649afd701df3ae3b"}
Nov 03 20:43:42.827441 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.827421    5915 generic.go:356] "Generic (PLEG): container finished" podID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerID="d9b182b9233825b7bc94ae63d60119bcd3cd165d2bb30e7937c57288fa685724" exitCode=0
Nov 03 20:43:42.827527 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.827502    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerDied","Data":"d9b182b9233825b7bc94ae63d60119bcd3cd165d2bb30e7937c57288fa685724"}
Nov 03 20:43:42.829024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.829002    5915 generic.go:356] "Generic (PLEG): container finished" podID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerID="f859aeba3ae26c3a1d7cadbe4b54898c1422def13e54d95f072aa03efd4662bc" exitCode=0
Nov 03 20:43:42.829111 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.829078    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerDied","Data":"f859aeba3ae26c3a1d7cadbe4b54898c1422def13e54d95f072aa03efd4662bc"}
Nov 03 20:43:42.830754 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.830734    5915 generic.go:356] "Generic (PLEG): container finished" podID="804f327c-7502-47da-9bff-03021ab05656" containerID="2e0fb6ff6f471bda7becba78b06965190588f02f89fc702886a0b1c4b9302c07" exitCode=0
Nov 03 20:43:42.830822 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.830800    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9" event={"ID":"804f327c-7502-47da-9bff-03021ab05656","Type":"ContainerDied","Data":"2e0fb6ff6f471bda7becba78b06965190588f02f89fc702886a0b1c4b9302c07"}
Nov 03 20:43:42.832674 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.832389    5915 generic.go:356] "Generic (PLEG): container finished" podID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerID="7392d821f9deef58bb7e5466191ecadd3de49ce89fd35d222c40e5733801b518" exitCode=0
Nov 03 20:43:42.832674 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.832478    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerDied","Data":"7392d821f9deef58bb7e5466191ecadd3de49ce89fd35d222c40e5733801b518"}
Nov 03 20:43:42.834135 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.834115    5915 generic.go:356] "Generic (PLEG): container finished" podID="bdda7ae6-846d-4865-9040-8042e6348844" containerID="333b0a25b971570b2e278763abe57a64023abf2b61094c4bad71c11f6f2e23bb" exitCode=0
Nov 03 20:43:42.834211 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:42.834174    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz" event={"ID":"bdda7ae6-846d-4865-9040-8042e6348844","Type":"ContainerDied","Data":"333b0a25b971570b2e278763abe57a64023abf2b61094c4bad71c11f6f2e23bb"}
Nov 03 20:43:43.843023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.842987    5915 generic.go:356] "Generic (PLEG): container finished" podID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerID="1a2f318f0fd6227218e0c47588495bccafe8035e1cf8a0e768231610c15e6c13" exitCode=0
Nov 03 20:43:43.843390 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.843072    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl" event={"ID":"55ffa83a-3863-4f0b-aa02-2988bf877e98","Type":"ContainerDied","Data":"1a2f318f0fd6227218e0c47588495bccafe8035e1cf8a0e768231610c15e6c13"}
Nov 03 20:43:43.847062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.847036    5915 generic.go:356] "Generic (PLEG): container finished" podID="a687e327-f01a-457b-ab4e-7be9f827d493" containerID="2a15555c56f0d9b00f9e01d7f0952e6dcd7766695e9e95ee34c8d3dd93d6496d" exitCode=0
Nov 03 20:43:43.847184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.847127    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz" event={"ID":"a687e327-f01a-457b-ab4e-7be9f827d493","Type":"ContainerDied","Data":"2a15555c56f0d9b00f9e01d7f0952e6dcd7766695e9e95ee34c8d3dd93d6496d"}
Nov 03 20:43:43.852840 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.850591    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerStarted","Data":"df1f4f3aaadcca3dd5f689d708bd0a03cc0ccee2c2b756bc320d9142e5102cca"}
Nov 03 20:43:43.853663 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.853640    5915 generic.go:356] "Generic (PLEG): container finished" podID="e2233017-06ef-4c6e-bece-94c4128c467d" containerID="ac351f4ccfaf6d69d685df7808457f4be8deeb45da27e2e30a924a488ad1a5b8" exitCode=0
Nov 03 20:43:43.853737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.853719    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw" event={"ID":"e2233017-06ef-4c6e-bece-94c4128c467d","Type":"ContainerDied","Data":"ac351f4ccfaf6d69d685df7808457f4be8deeb45da27e2e30a924a488ad1a5b8"}
Nov 03 20:43:43.856489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.856405    5915 generic.go:356] "Generic (PLEG): container finished" podID="992a347e-1791-4721-8260-9665ec97e3bd" containerID="f1c90d39bf07ac6467d4e7b431b0fc159fd478dbd05df3b1667c515e8a4b5a90" exitCode=0
Nov 03 20:43:43.856489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:43.856459    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x" event={"ID":"992a347e-1791-4721-8260-9665ec97e3bd","Type":"ContainerDied","Data":"f1c90d39bf07ac6467d4e7b431b0fc159fd478dbd05df3b1667c515e8a4b5a90"}
Nov 03 20:43:44.867170 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.867132    5915 generic.go:356] "Generic (PLEG): container finished" podID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerID="df1f4f3aaadcca3dd5f689d708bd0a03cc0ccee2c2b756bc320d9142e5102cca" exitCode=0
Nov 03 20:43:44.867580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.867252    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerDied","Data":"df1f4f3aaadcca3dd5f689d708bd0a03cc0ccee2c2b756bc320d9142e5102cca"}
Nov 03 20:43:44.875088 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.875060    5915 generic.go:356] "Generic (PLEG): container finished" podID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerID="6aa44750bae73f9672f0eaf535dcb9a902895a0ca530fcdfa812b3ee2c682a3d" exitCode=0
Nov 03 20:43:44.875198 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.875172    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerDied","Data":"6aa44750bae73f9672f0eaf535dcb9a902895a0ca530fcdfa812b3ee2c682a3d"}
Nov 03 20:43:44.881440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.881414    5915 generic.go:356] "Generic (PLEG): container finished" podID="bdda7ae6-846d-4865-9040-8042e6348844" containerID="38606d1e135c0e5d370645c0d93a667fc9fa01d6d308169779a724b12f0d0257" exitCode=0
Nov 03 20:43:44.881541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.881469    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz" event={"ID":"bdda7ae6-846d-4865-9040-8042e6348844","Type":"ContainerDied","Data":"38606d1e135c0e5d370645c0d93a667fc9fa01d6d308169779a724b12f0d0257"}
Nov 03 20:43:44.999934 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:44.999896    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:46.043821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:46.043785    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:43:50.979233 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:50.979195    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerStarted","Data":"4729091712d9d691b2541f713b66ff48c38061b95a612d2b69c3d5a1390e9ab0"}
Nov 03 20:43:52.001149 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:52.001107    5915 generic.go:356] "Generic (PLEG): container finished" podID="804f327c-7502-47da-9bff-03021ab05656" containerID="3fe246551948d9e99118c8ecd13f607e6ba8638c08098d166e3fe6ba6a927024" exitCode=0
Nov 03 20:43:52.001683 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:52.001215    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9" event={"ID":"804f327c-7502-47da-9bff-03021ab05656","Type":"ContainerDied","Data":"3fe246551948d9e99118c8ecd13f607e6ba8638c08098d166e3fe6ba6a927024"}
Nov 03 20:43:52.015010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:52.013979    5915 generic.go:356] "Generic (PLEG): container finished" podID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerID="4729091712d9d691b2541f713b66ff48c38061b95a612d2b69c3d5a1390e9ab0" exitCode=0
Nov 03 20:43:52.015010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:43:52.014821    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerDied","Data":"4729091712d9d691b2541f713b66ff48c38061b95a612d2b69c3d5a1390e9ab0"}
Nov 03 20:44:04.198676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:04.197968    5915 generic.go:356] "Generic (PLEG): container finished" podID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerID="a8e7e8194d5ea8481488ffd978b51a9e88d48dd999f7f9c615e5fb4935bca794" exitCode=0
Nov 03 20:44:04.198676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:04.198146    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982" event={"ID":"ba3612bf-03b4-4a45-a621-77bde7cb7abf","Type":"ContainerDied","Data":"a8e7e8194d5ea8481488ffd978b51a9e88d48dd999f7f9c615e5fb4935bca794"}
Nov 03 20:44:05.219649 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.218684    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerStarted","Data":"53a5264021c2cd167ad5a6675f6ce775ea1ff01edbff04d188fcf5af714f2752"}
Nov 03 20:44:05.462030 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.461975    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:44:05.580809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.580699    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" podStartSLOduration=2.662814226 podStartE2EDuration="39.580678853s" totalImagesPullingTime="36.917864627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:43:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:05.369408493 +0000 UTC m=+259.820777670" watchObservedRunningTime="2026-11-03 20:44:05.580678853 +0000 UTC m=+260.032048030"
Nov 03 20:44:05.644127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.644088    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\") pod \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") "
Nov 03 20:44:05.644317 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.644203    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\" (UniqueName: \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\") pod \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") "
Nov 03 20:44:05.644317 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.644241    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\") pod \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\" (UID: \"ba3612bf-03b4-4a45-a621-77bde7cb7abf\") "
Nov 03 20:44:05.647193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.647131    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle" pod "ba3612bf-03b4-4a45-a621-77bde7cb7abf" (UID: "ba3612bf-03b4-4a45-a621-77bde7cb7abf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:05.650709 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.650664    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw" pod "ba3612bf-03b4-4a45-a621-77bde7cb7abf" (UID: "ba3612bf-03b4-4a45-a621-77bde7cb7abf"). InnerVolumeSpecName "kube-api-access-qrpkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:05.652501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.652477    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util" pod "ba3612bf-03b4-4a45-a621-77bde7cb7abf" (UID: "ba3612bf-03b4-4a45-a621-77bde7cb7abf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:05.745356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.745323    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrpkw\" (UniqueName: \"kubernetes.io/projected/ba3612bf-03b4-4a45-a621-77bde7cb7abf-kube-api-access-qrpkw\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:05.745356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.745359    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:05.745656 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:05.745376    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba3612bf-03b4-4a45-a621-77bde7cb7abf-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:06.233726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:06.232946    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982" event={"ID":"ba3612bf-03b4-4a45-a621-77bde7cb7abf","Type":"ContainerDied","Data":"0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796"}
Nov 03 20:44:06.233726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:06.233042    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0601ae9e76eb8dc50318cb4b8045c3b12b55c2efd2c7ad3a509aba48dc0a0796"
Nov 03 20:44:06.233726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:06.233133    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c67b982"
Nov 03 20:44:06.239154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:06.238880    5915 generic.go:356] "Generic (PLEG): container finished" podID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerID="53a5264021c2cd167ad5a6675f6ce775ea1ff01edbff04d188fcf5af714f2752" exitCode=0
Nov 03 20:44:06.239154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:06.238956    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerDied","Data":"53a5264021c2cd167ad5a6675f6ce775ea1ff01edbff04d188fcf5af714f2752"}
Nov 03 20:44:07.013706 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:44:07.013597    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:44:07.248132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.247823    5915 generic.go:356] "Generic (PLEG): container finished" podID="992a347e-1791-4721-8260-9665ec97e3bd" containerID="fe954fa95f9bb0a5a61c55959522ba233053850420b561ec8b5522e64751452a" exitCode=0
Nov 03 20:44:07.248132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.247932    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x" event={"ID":"992a347e-1791-4721-8260-9665ec97e3bd","Type":"ContainerDied","Data":"fe954fa95f9bb0a5a61c55959522ba233053850420b561ec8b5522e64751452a"}
Nov 03 20:44:07.251897 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.251648    5915 generic.go:356] "Generic (PLEG): container finished" podID="804f327c-7502-47da-9bff-03021ab05656" containerID="51d7874c356de795b008e4facc8e6b2a6e2915446cddf13d2869ea6c03ed6e66" exitCode=0
Nov 03 20:44:07.251897 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.251720    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9" event={"ID":"804f327c-7502-47da-9bff-03021ab05656","Type":"ContainerDied","Data":"51d7874c356de795b008e4facc8e6b2a6e2915446cddf13d2869ea6c03ed6e66"}
Nov 03 20:44:07.255443 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.254842    5915 generic.go:356] "Generic (PLEG): container finished" podID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerID="98c29453a4e3c4c160ce81b5da10d140c90fd72be50b4cf11bdbc8737f4aaeeb" exitCode=0
Nov 03 20:44:07.255443 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.255339    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerDied","Data":"98c29453a4e3c4c160ce81b5da10d140c90fd72be50b4cf11bdbc8737f4aaeeb"}
Nov 03 20:44:07.442155 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.442047    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:44:07.562158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.561523    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\") pod \"e1769b07-54e2-48fe-956c-98e1a8d79e39\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") "
Nov 03 20:44:07.562158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.561585    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\") pod \"e1769b07-54e2-48fe-956c-98e1a8d79e39\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") "
Nov 03 20:44:07.562158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.561726    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\" (UniqueName: \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\") pod \"e1769b07-54e2-48fe-956c-98e1a8d79e39\" (UID: \"e1769b07-54e2-48fe-956c-98e1a8d79e39\") "
Nov 03 20:44:07.563871 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.563842    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle" pod "e1769b07-54e2-48fe-956c-98e1a8d79e39" (UID: "e1769b07-54e2-48fe-956c-98e1a8d79e39"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:07.564452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.564403    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2" pod "e1769b07-54e2-48fe-956c-98e1a8d79e39" (UID: "e1769b07-54e2-48fe-956c-98e1a8d79e39"). InnerVolumeSpecName "kube-api-access-64zr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:07.570038 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.570014    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util" pod "e1769b07-54e2-48fe-956c-98e1a8d79e39" (UID: "e1769b07-54e2-48fe-956c-98e1a8d79e39"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:07.663050 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.663011    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64zr2\" (UniqueName: \"kubernetes.io/projected/e1769b07-54e2-48fe-956c-98e1a8d79e39-kube-api-access-64zr2\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:07.663050 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.663039    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:07.663150 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.663056    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e1769b07-54e2-48fe-956c-98e1a8d79e39-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:07.833938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:07.833852    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:08.261505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.261478    5915 generic.go:356] "Generic (PLEG): container finished" podID="bdda7ae6-846d-4865-9040-8042e6348844" containerID="7b8eaec3befc751116627844f72212a6e93cab1c7b6db7c2068161f0cb007579" exitCode=0
Nov 03 20:44:08.261918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.261582    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz" event={"ID":"bdda7ae6-846d-4865-9040-8042e6348844","Type":"ContainerDied","Data":"7b8eaec3befc751116627844f72212a6e93cab1c7b6db7c2068161f0cb007579"}
Nov 03 20:44:08.263713 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.263690    5915 generic.go:356] "Generic (PLEG): container finished" podID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerID="c822d8bcfbdab0f174f269fb6855a1737a44fa772fe2e1a3d4e488df7af7536d" exitCode=0
Nov 03 20:44:08.263820 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.263772    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl" event={"ID":"55ffa83a-3863-4f0b-aa02-2988bf877e98","Type":"ContainerDied","Data":"c822d8bcfbdab0f174f269fb6855a1737a44fa772fe2e1a3d4e488df7af7536d"}
Nov 03 20:44:08.265843 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.265823    5915 generic.go:356] "Generic (PLEG): container finished" podID="a687e327-f01a-457b-ab4e-7be9f827d493" containerID="2ea062d0ca8dbf4315faaf3d923f146fa735b0d1d53e19d9a1ef019c4bad57e7" exitCode=0
Nov 03 20:44:08.265975 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.265903    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz" event={"ID":"a687e327-f01a-457b-ab4e-7be9f827d493","Type":"ContainerDied","Data":"2ea062d0ca8dbf4315faaf3d923f146fa735b0d1d53e19d9a1ef019c4bad57e7"}
Nov 03 20:44:08.267983 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.267963    5915 generic.go:356] "Generic (PLEG): container finished" podID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerID="d271bcb416e115765f7fdc4a717890ed9090835904fe9630c6eacef8226a7566" exitCode=0
Nov 03 20:44:08.268091 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.268050    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerDied","Data":"d271bcb416e115765f7fdc4a717890ed9090835904fe9630c6eacef8226a7566"}
Nov 03 20:44:08.270020 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.270002    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p" event={"ID":"e1769b07-54e2-48fe-956c-98e1a8d79e39","Type":"ContainerDied","Data":"79b4162ec3b329b13a679ac522d11fc1f3fe4b8aea0f9282f200069b743b0cfd"}
Nov 03 20:44:08.270105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.270041    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79b4162ec3b329b13a679ac522d11fc1f3fe4b8aea0f9282f200069b743b0cfd"
Nov 03 20:44:08.270105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.270006    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6jfv2p"
Nov 03 20:44:08.271643 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.271627    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:44:08.272454 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.272435    5915 generic.go:356] "Generic (PLEG): container finished" podID="e2233017-06ef-4c6e-bece-94c4128c467d" containerID="a1cf925e1172af9c890468dba848bd3cb7d24c785ec9dea319faddba52d4697a" exitCode=1
Nov 03 20:44:08.272543 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.272460    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw" event={"ID":"e2233017-06ef-4c6e-bece-94c4128c467d","Type":"ContainerDied","Data":"a1cf925e1172af9c890468dba848bd3cb7d24c785ec9dea319faddba52d4697a"}
Nov 03 20:44:08.274530 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.274509    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerStarted","Data":"9ae0e3de666d5c3dd2c0378180e0f2fa0128820e7c341edc9cf0fd6e1d0c4952"}
Nov 03 20:44:08.413681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.413653    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:44:08.519765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.519743    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:44:08.522656 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.522635    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:44:08.570120 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.570100    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\" (UniqueName: \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\") pod \"804f327c-7502-47da-9bff-03021ab05656\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") "
Nov 03 20:44:08.570220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.570156    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\") pod \"804f327c-7502-47da-9bff-03021ab05656\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") "
Nov 03 20:44:08.570220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.570183    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\") pod \"804f327c-7502-47da-9bff-03021ab05656\" (UID: \"804f327c-7502-47da-9bff-03021ab05656\") "
Nov 03 20:44:08.570853 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.570825    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle" pod "804f327c-7502-47da-9bff-03021ab05656" (UID: "804f327c-7502-47da-9bff-03021ab05656"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.572291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.572271    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5" pod "804f327c-7502-47da-9bff-03021ab05656" (UID: "804f327c-7502-47da-9bff-03021ab05656"). InnerVolumeSpecName "kube-api-access-2q8f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:08.575311 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.575290    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util" pod "804f327c-7502-47da-9bff-03021ab05656" (UID: "804f327c-7502-47da-9bff-03021ab05656"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.664126 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.664048    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" podStartSLOduration=3.538681227 podStartE2EDuration="48.664036509s" totalImagesPullingTime="45.125355282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:43:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:08.663623281 +0000 UTC m=+263.114992458" watchObservedRunningTime="2026-11-03 20:44:08.664036509 +0000 UTC m=+263.115405685"
Nov 03 20:44:08.671042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671018    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\") pod \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") "
Nov 03 20:44:08.671138 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671065    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\" (UniqueName: \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\") pod \"992a347e-1791-4721-8260-9665ec97e3bd\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") "
Nov 03 20:44:08.671138 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671127    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\") pod \"992a347e-1791-4721-8260-9665ec97e3bd\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") "
Nov 03 20:44:08.671263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671144    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\") pod \"992a347e-1791-4721-8260-9665ec97e3bd\" (UID: \"992a347e-1791-4721-8260-9665ec97e3bd\") "
Nov 03 20:44:08.671263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671180    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\" (UniqueName: \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\") pod \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") "
Nov 03 20:44:08.671263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671205    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\") pod \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\" (UID: \"7387d9ec-bfa6-455d-99f8-ac98e19daf3a\") "
Nov 03 20:44:08.671413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671399    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2q8f5\" (UniqueName: \"kubernetes.io/projected/804f327c-7502-47da-9bff-03021ab05656-kube-api-access-2q8f5\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.671469 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671419    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.671469 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671432    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/804f327c-7502-47da-9bff-03021ab05656-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.671551 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671499    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle" pod "7387d9ec-bfa6-455d-99f8-ac98e19daf3a" (UID: "7387d9ec-bfa6-455d-99f8-ac98e19daf3a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.671902 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.671876    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle" pod "992a347e-1791-4721-8260-9665ec97e3bd" (UID: "992a347e-1791-4721-8260-9665ec97e3bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.673867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.673846    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479" pod "7387d9ec-bfa6-455d-99f8-ac98e19daf3a" (UID: "7387d9ec-bfa6-455d-99f8-ac98e19daf3a"). InnerVolumeSpecName "kube-api-access-27479". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:08.673867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.673853    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg" pod "992a347e-1791-4721-8260-9665ec97e3bd" (UID: "992a347e-1791-4721-8260-9665ec97e3bd"). InnerVolumeSpecName "kube-api-access-856tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:08.677570 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.677541    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util" pod "992a347e-1791-4721-8260-9665ec97e3bd" (UID: "992a347e-1791-4721-8260-9665ec97e3bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.677639 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.677609    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util" pod "7387d9ec-bfa6-455d-99f8-ac98e19daf3a" (UID: "7387d9ec-bfa6-455d-99f8-ac98e19daf3a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:08.771788 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771756    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.771788 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771787    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/992a347e-1791-4721-8260-9665ec97e3bd-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.771942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771797    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27479\" (UniqueName: \"kubernetes.io/projected/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-kube-api-access-27479\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.771942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771806    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.771942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771815    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7387d9ec-bfa6-455d-99f8-ac98e19daf3a-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:08.771942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:08.771824    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-856tg\" (UniqueName: \"kubernetes.io/projected/992a347e-1791-4721-8260-9665ec97e3bd-kube-api-access-856tg\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.282044    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x" event={"ID":"992a347e-1791-4721-8260-9665ec97e3bd","Type":"ContainerDied","Data":"db3e053f1c07ba76bfbb010ed417a9f3e8941ee65a117ebb9f1ecc625d01092f"}
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.282123    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cgld8x"
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.282138    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db3e053f1c07ba76bfbb010ed417a9f3e8941ee65a117ebb9f1ecc625d01092f"
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.284995    5915 generic.go:356] "Generic (PLEG): container finished" podID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerID="9ae0e3de666d5c3dd2c0378180e0f2fa0128820e7c341edc9cf0fd6e1d0c4952" exitCode=0
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.285112    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerDied","Data":"9ae0e3de666d5c3dd2c0378180e0f2fa0128820e7c341edc9cf0fd6e1d0c4952"}
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.287418    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9" event={"ID":"804f327c-7502-47da-9bff-03021ab05656","Type":"ContainerDied","Data":"3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae"}
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.287462    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e71541b6369122e6fc28050c331897c452e4948532c0dfa9a6b4c71238716ae"
Nov 03 20:44:09.288315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.287552    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cxfnh9"
Nov 03 20:44:09.290379 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.290351    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp" event={"ID":"7387d9ec-bfa6-455d-99f8-ac98e19daf3a","Type":"ContainerDied","Data":"f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5"}
Nov 03 20:44:09.290379 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.290377    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644qbjsp"
Nov 03 20:44:09.290594 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.290392    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f26cbe2f472653d8b711b700c90ad18185e6bfb5801b40754971c0ecd40de8e5"
Nov 03 20:44:09.447005 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.446982    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:44:09.535288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.535267    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:44:09.537524 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.537495    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:44:09.540348 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.540297    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:44:09.546394 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.546378    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:44:09.547142 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.547128    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:44:09.579987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.579969    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\" (UniqueName: \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\") pod \"55ffa83a-3863-4f0b-aa02-2988bf877e98\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") "
Nov 03 20:44:09.580080 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.580010    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\") pod \"55ffa83a-3863-4f0b-aa02-2988bf877e98\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") "
Nov 03 20:44:09.580169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.580152    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\") pod \"55ffa83a-3863-4f0b-aa02-2988bf877e98\" (UID: \"55ffa83a-3863-4f0b-aa02-2988bf877e98\") "
Nov 03 20:44:09.580414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.580396    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle" pod "55ffa83a-3863-4f0b-aa02-2988bf877e98" (UID: "55ffa83a-3863-4f0b-aa02-2988bf877e98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.582143 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.582116    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv" pod "55ffa83a-3863-4f0b-aa02-2988bf877e98" (UID: "55ffa83a-3863-4f0b-aa02-2988bf877e98"). InnerVolumeSpecName "kube-api-access-68hrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:09.585240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.585217    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util" pod "55ffa83a-3863-4f0b-aa02-2988bf877e98" (UID: "55ffa83a-3863-4f0b-aa02-2988bf877e98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.681608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681576    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\" (UniqueName: \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\") pod \"e5014d06-637a-4e17-b7d1-4790aebbdc36\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") "
Nov 03 20:44:09.681734 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681633    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\" (UniqueName: \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\") pod \"bdda7ae6-846d-4865-9040-8042e6348844\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") "
Nov 03 20:44:09.681734 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681667    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\") pod \"e2233017-06ef-4c6e-bece-94c4128c467d\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") "
Nov 03 20:44:09.681734 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681685    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\" (UniqueName: \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\") pod \"e2233017-06ef-4c6e-bece-94c4128c467d\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") "
Nov 03 20:44:09.681734 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681729    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\") pod \"e2233017-06ef-4c6e-bece-94c4128c467d\" (UID: \"e2233017-06ef-4c6e-bece-94c4128c467d\") "
Nov 03 20:44:09.681955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681764    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\") pod \"bdda7ae6-846d-4865-9040-8042e6348844\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") "
Nov 03 20:44:09.681955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.681812    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\" (UniqueName: \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\") pod \"a687e327-f01a-457b-ab4e-7be9f827d493\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") "
Nov 03 20:44:09.682498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682469    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle" pod "e2233017-06ef-4c6e-bece-94c4128c467d" (UID: "e2233017-06ef-4c6e-bece-94c4128c467d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.682608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682576    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\") pod \"e5014d06-637a-4e17-b7d1-4790aebbdc36\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") "
Nov 03 20:44:09.682684 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682620    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\") pod \"bdda7ae6-846d-4865-9040-8042e6348844\" (UID: \"bdda7ae6-846d-4865-9040-8042e6348844\") "
Nov 03 20:44:09.682684 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682655    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\") pod \"e5014d06-637a-4e17-b7d1-4790aebbdc36\" (UID: \"e5014d06-637a-4e17-b7d1-4790aebbdc36\") "
Nov 03 20:44:09.682808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682693    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\") pod \"a687e327-f01a-457b-ab4e-7be9f827d493\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") "
Nov 03 20:44:09.682808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682733    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\") pod \"a687e327-f01a-457b-ab4e-7be9f827d493\" (UID: \"a687e327-f01a-457b-ab4e-7be9f827d493\") "
Nov 03 20:44:09.682977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682926    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.682977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682946    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68hrv\" (UniqueName: \"kubernetes.io/projected/55ffa83a-3863-4f0b-aa02-2988bf877e98-kube-api-access-68hrv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.682977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.682960    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55ffa83a-3863-4f0b-aa02-2988bf877e98-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.684026 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.684002    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz" pod "e2233017-06ef-4c6e-bece-94c4128c467d" (UID: "e2233017-06ef-4c6e-bece-94c4128c467d"). InnerVolumeSpecName "kube-api-access-nk2nz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:09.684352 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.684319    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw" pod "e5014d06-637a-4e17-b7d1-4790aebbdc36" (UID: "e5014d06-637a-4e17-b7d1-4790aebbdc36"). InnerVolumeSpecName "kube-api-access-j8fnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:09.684448 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.684412    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle" pod "bdda7ae6-846d-4865-9040-8042e6348844" (UID: "bdda7ae6-846d-4865-9040-8042e6348844"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.685069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.685046    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz" pod "a687e327-f01a-457b-ab4e-7be9f827d493" (UID: "a687e327-f01a-457b-ab4e-7be9f827d493"). InnerVolumeSpecName "kube-api-access-gc5pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:09.685154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.685124    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle" pod "e5014d06-637a-4e17-b7d1-4790aebbdc36" (UID: "e5014d06-637a-4e17-b7d1-4790aebbdc36"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.685308 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.685294    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle" pod "a687e327-f01a-457b-ab4e-7be9f827d493" (UID: "a687e327-f01a-457b-ab4e-7be9f827d493"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.690440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.690413    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util" pod "e5014d06-637a-4e17-b7d1-4790aebbdc36" (UID: "e5014d06-637a-4e17-b7d1-4790aebbdc36"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.691602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.691576    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util" pod "e2233017-06ef-4c6e-bece-94c4128c467d" (UID: "e2233017-06ef-4c6e-bece-94c4128c467d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.692108 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.692083    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh" pod "bdda7ae6-846d-4865-9040-8042e6348844" (UID: "bdda7ae6-846d-4865-9040-8042e6348844"). InnerVolumeSpecName "kube-api-access-pbrjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:09.692542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.692513    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util" pod "bdda7ae6-846d-4865-9040-8042e6348844" (UID: "bdda7ae6-846d-4865-9040-8042e6348844"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.693297 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.693278    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util" pod "a687e327-f01a-457b-ab4e-7be9f827d493" (UID: "a687e327-f01a-457b-ab4e-7be9f827d493"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:09.783914 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783890    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.783914 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783910    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a687e327-f01a-457b-ab4e-7be9f827d493-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783920    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8fnw\" (UniqueName: \"kubernetes.io/projected/e5014d06-637a-4e17-b7d1-4790aebbdc36-kube-api-access-j8fnw\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783930    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbrjh\" (UniqueName: \"kubernetes.io/projected/bdda7ae6-846d-4865-9040-8042e6348844-kube-api-access-pbrjh\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783940    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783948    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk2nz\" (UniqueName: \"kubernetes.io/projected/e2233017-06ef-4c6e-bece-94c4128c467d-kube-api-access-nk2nz\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783956    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e2233017-06ef-4c6e-bece-94c4128c467d-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783963    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783971    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc5pz\" (UniqueName: \"kubernetes.io/projected/a687e327-f01a-457b-ab4e-7be9f827d493-kube-api-access-gc5pz\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783979    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783986    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bdda7ae6-846d-4865-9040-8042e6348844-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:09.784062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:09.783993    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5014d06-637a-4e17-b7d1-4790aebbdc36-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:10.296474 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.296444    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd"
Nov 03 20:44:10.296938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.296440    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71dvbpd" event={"ID":"e5014d06-637a-4e17-b7d1-4790aebbdc36","Type":"ContainerDied","Data":"17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83"}
Nov 03 20:44:10.296938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.296595    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17b7655c7da675750195ac3e0be735c3992d5e6b84714e23a1a94e445f584f83"
Nov 03 20:44:10.297830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.297812    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:44:10.298528 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.298512    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw"
Nov 03 20:44:10.298634 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.298532    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw" event={"ID":"e2233017-06ef-4c6e-bece-94c4128c467d","Type":"ContainerDied","Data":"19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10"}
Nov 03 20:44:10.298634 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.298578    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19e57bb0399b2202f1b21e7aab4706d7f8ba3d161ce9e793bda3871e85353f10"
Nov 03 20:44:10.300669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.300632    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz" event={"ID":"bdda7ae6-846d-4865-9040-8042e6348844","Type":"ContainerDied","Data":"7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654"}
Nov 03 20:44:10.300669 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.300655    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lq4qz"
Nov 03 20:44:10.300829 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.300675    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7621eaca7b992a5c7afef787b77f18d857ff2fe29b45abe837f05c7a08e6d654"
Nov 03 20:44:10.302481 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.302461    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl" event={"ID":"55ffa83a-3863-4f0b-aa02-2988bf877e98","Type":"ContainerDied","Data":"8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0"}
Nov 03 20:44:10.302481 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.302487    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d76fbdl"
Nov 03 20:44:10.302481 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.302492    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8dd73471185a2bc89c5778fb1006c2aa3fb28118adb78dfd3d043e085201e1f0"
Nov 03 20:44:10.305275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.305259    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz"
Nov 03 20:44:10.305374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.305284    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847apx7lz" event={"ID":"a687e327-f01a-457b-ab4e-7be9f827d493","Type":"ContainerDied","Data":"60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f"}
Nov 03 20:44:10.305374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.305322    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60d103a35d8c496df9cb0d45ffae46521e325b2fdfbc92951e97c80639a5bb4f"
Nov 03 20:44:10.432640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.432619    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:44:10.589697 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.589627    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\") pod \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") "
Nov 03 20:44:10.589841 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.589715    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\") pod \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") "
Nov 03 20:44:10.589841 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.589770    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\" (UniqueName: \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\") pod \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\" (UID: \"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5\") "
Nov 03 20:44:10.590206 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.590184    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle" pod "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" (UID: "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:10.592146 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.592120    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml" pod "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" (UID: "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5"). InnerVolumeSpecName "kube-api-access-bmmml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:10.594886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.594869    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util" pod "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" (UID: "5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:10.690682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.690652    5915 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:10.690682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.690677    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmmml\" (UniqueName: \"kubernetes.io/projected/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-kube-api-access-bmmml\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:10.690682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:10.690687    5915 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5-util\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:11.312811 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:11.312778    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb" event={"ID":"5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5","Type":"ContainerDied","Data":"b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b"}
Nov 03 20:44:11.313185 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:11.312831    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7f969557a2d24a821c3edbd7dafe8a2fd732cb12d336a6cbbc9f4f0e787ef0b"
Nov 03 20:44:11.313185 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:11.312790    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4ckqxxb"
Nov 03 20:44:13.324658 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:13.324627    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:13.540077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:13.540049    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:14.121077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.121045    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:14.420173 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420097    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"]
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420382    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420393    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420403    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420411    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420424    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420431    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420441    5915 cpu_manager.go:414] "containerMap: removing container" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420448    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420458    5915 cpu_manager.go:414] "containerMap: removing container" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420463    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420471    5915 cpu_manager.go:414] "containerMap: removing container" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420474    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420480    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420484    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420489    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420494    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420503    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420507    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420514    5915 cpu_manager.go:414] "containerMap: removing container" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420518    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="util"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420523    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420527    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420533    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420537    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420543    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420547    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420552    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420570    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="extract"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420579    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="pull"
Nov 03 20:44:14.420544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420583    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420589    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420594    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420599    5915 cpu_manager.go:414] "containerMap: removing container" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420603    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420612    5915 cpu_manager.go:414] "containerMap: removing container" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420616    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420628    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420632    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420637    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420641    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420648    5915 cpu_manager.go:414] "containerMap: removing container" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420652    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420658    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420662    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420667    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420672    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420677    5915 cpu_manager.go:414] "containerMap: removing container" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420680    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420686    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420689    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420695    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420698    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420704    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420708    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="util"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420712    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420716    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420722    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420725    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420731    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420735    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420741    5915 cpu_manager.go:414] "containerMap: removing container" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420744    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420751    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420756    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="pull"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420761    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420766    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420828    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a687e327-f01a-457b-ab4e-7be9f827d493" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420836    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="55ffa83a-3863-4f0b-aa02-2988bf877e98" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420840    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba3612bf-03b4-4a45-a621-77bde7cb7abf" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420846    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7387d9ec-bfa6-455d-99f8-ac98e19daf3a" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420852    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdda7ae6-846d-4865-9040-8042e6348844" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420857    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2233017-06ef-4c6e-bece-94c4128c467d" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420863    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1769b07-54e2-48fe-956c-98e1a8d79e39" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420868    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="804f327c-7502-47da-9bff-03021ab05656" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420873    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="5efdc9bf-dcc5-403d-8cf7-2cf370cab9a5" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420878    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5014d06-637a-4e17-b7d1-4790aebbdc36" containerName="extract"
Nov 03 20:44:14.421426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.420885    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="992a347e-1791-4721-8260-9665ec97e3bd" containerName="extract"
Nov 03 20:44:14.444223 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.444197    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.498374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498341    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 20:44:14.498374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498346    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 20:44:14.498616 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498553    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-7q7qx\""
Nov 03 20:44:14.498658 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498616    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 20:44:14.498755 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498740    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 20:44:14.498808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.498744    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 20:44:14.517053 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517023    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll45l\" (UniqueName: \"kubernetes.io/projected/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-kube-api-access-ll45l\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.517219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517126    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.517219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517159    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.517335 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517227    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.517335 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517263    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.517402 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.517365    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.550502 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.550473    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"]
Nov 03 20:44:14.583467 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.583435    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:14.617782 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617747    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.617972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617795    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll45l\" (UniqueName: \"kubernetes.io/projected/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-kube-api-access-ll45l\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.617972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617851    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.617972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617876    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.617972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617906    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.617972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.617929    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.618281 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.618243    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.620381 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.620344    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.620510 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.620413    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.620606 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.620586    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.620675 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.620628    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.681396 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.681320    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll45l\" (UniqueName: \"kubernetes.io/projected/8b02ab99-ab6c-4e91-ac4d-b729ecaf8481-kube-api-access-ll45l\") pod \"opendatahub-operator-controller-manager-78789f7d7-hffr4\" (UID: \"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.753432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.753398    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:14.795422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.795381    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:14.965253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:14.965222    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:15.306669 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:15.306639    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b02ab99_ab6c_4e91_ac4d_b729ecaf8481.slice/crio-7f2c2282a648f4dc9651f1fe1b8d29a3a6781b348ec12bd6b55dca4694a3c031 WatchSource:0}: Error finding container 7f2c2282a648f4dc9651f1fe1b8d29a3a6781b348ec12bd6b55dca4694a3c031: Status 404 returned error can't find the container with id 7f2c2282a648f4dc9651f1fe1b8d29a3a6781b348ec12bd6b55dca4694a3c031
Nov 03 20:44:15.306992 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:15.306971    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"]
Nov 03 20:44:15.335426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:15.335395    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4" event={"ID":"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481","Type":"ContainerStarted","Data":"7f2c2282a648f4dc9651f1fe1b8d29a3a6781b348ec12bd6b55dca4694a3c031"}
Nov 03 20:44:16.187143 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:16.187115    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:16.496829 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:16.496785    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:17.634271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:17.634226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:17.844454 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:17.844395    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:18.159113 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:18.159078    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:19.356059 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:19.356027    5915 generic.go:356] "Generic (PLEG): container finished" podID="8b02ab99-ab6c-4e91-ac4d-b729ecaf8481" containerID="0f98d956d9bf123b0e9cf4e3d275fe5e05fb6e33f2b7562578e0ca75e1e957a9" exitCode=0
Nov 03 20:44:19.356397 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:19.356111    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4" event={"ID":"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481","Type":"ContainerDied","Data":"0f98d956d9bf123b0e9cf4e3d275fe5e05fb6e33f2b7562578e0ca75e1e957a9"}
Nov 03 20:44:20.053019 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.052982    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"]
Nov 03 20:44:20.055176 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.055160    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.064498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.064480    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d015d78-fc14-483c-acf8-15e64ea48871-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.064606 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.064515    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bkk9\" (UniqueName: \"kubernetes.io/projected/5d015d78-fc14-483c-acf8-15e64ea48871-kube-api-access-7bkk9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.114885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.114865    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:44:20.114885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.114865    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 03 20:44:20.114993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.114926    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xtwtz\""
Nov 03 20:44:20.126444 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.126422    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"]
Nov 03 20:44:20.165153 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.165133    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d015d78-fc14-483c-acf8-15e64ea48871-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.165213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.165171    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bkk9\" (UniqueName: \"kubernetes.io/projected/5d015d78-fc14-483c-acf8-15e64ea48871-kube-api-access-7bkk9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.165535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.165517    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5d015d78-fc14-483c-acf8-15e64ea48871-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.240819 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.240787    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bkk9\" (UniqueName: \"kubernetes.io/projected/5d015d78-fc14-483c-acf8-15e64ea48871-kube-api-access-7bkk9\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7snl2\" (UID: \"5d015d78-fc14-483c-acf8-15e64ea48871\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.363130 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.363052    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4" event={"ID":"8b02ab99-ab6c-4e91-ac4d-b729ecaf8481","Type":"ContainerStarted","Data":"412353f038693d52a28c1a4f013c680aeba6e933b8a14c10ced014a56ab5ccb4"}
Nov 03 20:44:20.363485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.363155    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:20.365404 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.365386    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"
Nov 03 20:44:20.523855 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.523811    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4" podStartSLOduration=1.776171174 podStartE2EDuration="6.52379809s" totalImagesPullingTime="4.259648433s" totalInitContainerRuntime="487.978483ms" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:20.523353492 +0000 UTC m=+274.974722681" watchObservedRunningTime="2026-11-03 20:44:20.52379809 +0000 UTC m=+274.975167274"
Nov 03 20:44:20.813320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:20.813294    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2"]
Nov 03 20:44:20.815980 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:20.815944    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d015d78_fc14_483c_acf8_15e64ea48871.slice/crio-c22ea48512c9ccb4e0ba3ce49a01d5d6c03b5ae50a09d2a81bc8836e75490510 WatchSource:0}: Error finding container c22ea48512c9ccb4e0ba3ce49a01d5d6c03b5ae50a09d2a81bc8836e75490510: Status 404 returned error can't find the container with id c22ea48512c9ccb4e0ba3ce49a01d5d6c03b5ae50a09d2a81bc8836e75490510
Nov 03 20:44:21.373602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:21.373544    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2" event={"ID":"5d015d78-fc14-483c-acf8-15e64ea48871","Type":"ContainerStarted","Data":"c22ea48512c9ccb4e0ba3ce49a01d5d6c03b5ae50a09d2a81bc8836e75490510"}
Nov 03 20:44:21.403505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:21.403478    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:21.652995 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:21.652920    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:23.593031 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:23.592941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:23.901709 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:23.901627    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:24.152228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.152164    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"]
Nov 03 20:44:24.158605 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.158583    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.197648 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.197612    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8x7f\" (UniqueName: \"kubernetes.io/projected/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-kube-api-access-n8x7f\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.197648 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.197646    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-tmp\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.214129 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.214093    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-2bffb\""
Nov 03 20:44:24.264104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.264054    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 03 20:44:24.264104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.264054    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:44:24.266001 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.265974    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"]
Nov 03 20:44:24.298857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.298823    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8x7f\" (UniqueName: \"kubernetes.io/projected/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-kube-api-access-n8x7f\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.298857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.298856    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-tmp\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.299185 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.299164    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-tmp\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.361698 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.361665    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8x7f\" (UniqueName: \"kubernetes.io/projected/1dc5cfdb-4dca-4bf1-9471-29f4745ecee2-kube-api-access-n8x7f\") pod \"jobset-operator-674cd59dc5-8sgp5\" (UID: \"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.390012 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.389983    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2" event={"ID":"5d015d78-fc14-483c-acf8-15e64ea48871","Type":"ContainerStarted","Data":"75982409a995dd595728b65cabe27c04cecfb6e6bfabea944a6388c5ffe5eb39"}
Nov 03 20:44:24.484993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.484965    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"
Nov 03 20:44:24.531831 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.531781    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7snl2" podStartSLOduration=3.02381693 podStartE2EDuration="5.531766842s" totalImagesPullingTime="2.507949912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:24.530053082 +0000 UTC m=+278.981422258" watchObservedRunningTime="2026-11-03 20:44:24.531766842 +0000 UTC m=+278.983136019"
Nov 03 20:44:24.884002 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:24.883786    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5"]
Nov 03 20:44:24.886589 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:24.886531    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dc5cfdb_4dca_4bf1_9471_29f4745ecee2.slice/crio-4eb8c315e2c4ddc3d1d8f48daa554f8de649f14f6f3a5335d7c4031287c4e5e3 WatchSource:0}: Error finding container 4eb8c315e2c4ddc3d1d8f48daa554f8de649f14f6f3a5335d7c4031287c4e5e3: Status 404 returned error can't find the container with id 4eb8c315e2c4ddc3d1d8f48daa554f8de649f14f6f3a5335d7c4031287c4e5e3
Nov 03 20:44:25.396574 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:25.396528    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5" event={"ID":"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2","Type":"ContainerStarted","Data":"4eb8c315e2c4ddc3d1d8f48daa554f8de649f14f6f3a5335d7c4031287c4e5e3"}
Nov 03 20:44:26.024945 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:26.024907    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:26.259815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:26.259784    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:27.355889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.355853    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"]
Nov 03 20:44:27.359774 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.359740    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.422040 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.422004    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.422241 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.422086    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjm22\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-kube-api-access-wjm22\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.423515 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.423489    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"]
Nov 03 20:44:27.470592 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.470539    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 03 20:44:27.470867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.470832    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-th4rh\""
Nov 03 20:44:27.470984 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.470930    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 03 20:44:27.522917 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.522881    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.523081 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.522961    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjm22\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-kube-api-access-wjm22\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.585007 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.584978    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjm22\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-kube-api-access-wjm22\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.585289 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.585265    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-8pd8w\" (UID: \"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:27.674784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:27.674743    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:28.088619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:28.088491    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"]
Nov 03 20:44:28.091393 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:28.091343    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d6c0c29_53e0_41e7_bc60_1f6cd0d7365e.slice/crio-b0563e71a9741c719ba6bf1a5e38180295dd9fcd9f10f73815f3aecdd8589252 WatchSource:0}: Error finding container b0563e71a9741c719ba6bf1a5e38180295dd9fcd9f10f73815f3aecdd8589252: Status 404 returned error can't find the container with id b0563e71a9741c719ba6bf1a5e38180295dd9fcd9f10f73815f3aecdd8589252
Nov 03 20:44:28.416494 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:28.416401    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:28.417278 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:28.417247    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5" event={"ID":"1dc5cfdb-4dca-4bf1-9471-29f4745ecee2","Type":"ContainerStarted","Data":"deb82211a963917368fc3783b3056fce841c2dce8607c0e2864fe695f1cb617d"}
Nov 03 20:44:28.418698 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:28.418674    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w" event={"ID":"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e","Type":"ContainerStarted","Data":"b0563e71a9741c719ba6bf1a5e38180295dd9fcd9f10f73815f3aecdd8589252"}
Nov 03 20:44:28.536892 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:28.536843    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-8sgp5" podStartSLOduration=1.8370370249999999 podStartE2EDuration="4.536830146s" totalImagesPullingTime="2.699793121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:28.535505461 +0000 UTC m=+282.986874639" watchObservedRunningTime="2026-11-03 20:44:28.536830146 +0000 UTC m=+282.988199323"
Nov 03 20:44:29.098907 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:29.098870    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:31.285863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:31.285795    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:31.376192 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:31.376161    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-hffr4"
Nov 03 20:44:32.290027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:32.289994    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:32.442047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:32.442008    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w" event={"ID":"4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e","Type":"ContainerStarted","Data":"08d8bb3ae8b3bd9b7845d83b29298da8adde2305806f2e335ffc3a402322fb7c"}
Nov 03 20:44:32.442223 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:32.442151    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:32.567150 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:32.567059    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w" podStartSLOduration=1.77820306 podStartE2EDuration="5.567044977s" totalImagesPullingTime="3.788841917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:32.565369286 +0000 UTC m=+287.016738464" watchObservedRunningTime="2026-11-03 20:44:32.567044977 +0000 UTC m=+287.018414148"
Nov 03 20:44:33.505654 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:33.505612    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:35.504206 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:35.504176    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:35.876721 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:35.876639    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:36.803468 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.803438    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-86pkk"]
Nov 03 20:44:36.806886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.806867    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:36.867013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.866980    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-86pkk"]
Nov 03 20:44:36.899418 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.899392    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drvz5\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-kube-api-access-drvz5\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:36.899587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.899463    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-bound-sa-token\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:36.910202 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.910173    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:36.914168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:36.914153    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sxhsz\""
Nov 03 20:44:37.000126 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.000095    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-bound-sa-token\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:37.000286 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.000144    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drvz5\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-kube-api-access-drvz5\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:37.001672 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:44:37.001643    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:44:37.063657 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.063605    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-bound-sa-token\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:37.065008 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.064990    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drvz5\" (UniqueName: \"kubernetes.io/projected/2d785393-5d7e-46ef-bf32-22d9faeffadc-kube-api-access-drvz5\") pod \"cert-manager-597b95cb84-86pkk\" (UID: \"2d785393-5d7e-46ef-bf32-22d9faeffadc\") " pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:37.119141 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.119115    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-86pkk"
Nov 03 20:44:37.493501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:37.493385    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-86pkk"]
Nov 03 20:44:37.495609 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:37.495582    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d785393_5d7e_46ef_bf32_22d9faeffadc.slice/crio-4737b56f3af9e1324f4d53a610a6d1a3391e7982902c15b509c8d9fcd630c836 WatchSource:0}: Error finding container 4737b56f3af9e1324f4d53a610a6d1a3391e7982902c15b509c8d9fcd630c836: Status 404 returned error can't find the container with id 4737b56f3af9e1324f4d53a610a6d1a3391e7982902c15b509c8d9fcd630c836
Nov 03 20:44:38.449605 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:38.449549    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w"
Nov 03 20:44:38.471732 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:38.471698    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-86pkk" event={"ID":"2d785393-5d7e-46ef-bf32-22d9faeffadc","Type":"ContainerStarted","Data":"732b1eb1e361501c8ec70815f64409bc68597f36004de7c63b00a01ef481fdf9"}
Nov 03 20:44:38.471874 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:38.471739    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-86pkk" event={"ID":"2d785393-5d7e-46ef-bf32-22d9faeffadc","Type":"ContainerStarted","Data":"4737b56f3af9e1324f4d53a610a6d1a3391e7982902c15b509c8d9fcd630c836"}
Nov 03 20:44:38.707100 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:38.707004    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-86pkk" podStartSLOduration=2.706990634 podStartE2EDuration="2.706990634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:38.705136396 +0000 UTC m=+293.156505574" watchObservedRunningTime="2026-11-03 20:44:38.706990634 +0000 UTC m=+293.158359808"
Nov 03 20:44:39.322627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:39.322587    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:39.602886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:39.602771    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:41.009472 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.009418    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:41.188077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.188029    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:41.469142 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.469113    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"]
Nov 03 20:44:41.472804 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.472786    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:41.534256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.534219    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwjs\" (UniqueName: \"kubernetes.io/projected/4aa1413a-79c3-48a2-816b-71790bbb6727-kube-api-access-nvwjs\") pod \"limitador-operator-controller-manager-5778bb4449-zxvm4\" (UID: \"4aa1413a-79c3-48a2-816b-71790bbb6727\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:41.580218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.580194    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 20:44:41.580372 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.580245    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 20:44:41.580372 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.580350    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-79svg\""
Nov 03 20:44:41.582454 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.582435    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"]
Nov 03 20:44:41.635449 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.635423    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwjs\" (UniqueName: \"kubernetes.io/projected/4aa1413a-79c3-48a2-816b-71790bbb6727-kube-api-access-nvwjs\") pod \"limitador-operator-controller-manager-5778bb4449-zxvm4\" (UID: \"4aa1413a-79c3-48a2-816b-71790bbb6727\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:41.699961 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.699930    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwjs\" (UniqueName: \"kubernetes.io/projected/4aa1413a-79c3-48a2-816b-71790bbb6727-kube-api-access-nvwjs\") pod \"limitador-operator-controller-manager-5778bb4449-zxvm4\" (UID: \"4aa1413a-79c3-48a2-816b-71790bbb6727\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:41.786055 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.786000    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:41.882624 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:41.882589    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:42.166708 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:42.166636    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"]
Nov 03 20:44:42.169519 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:42.169480    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa1413a_79c3_48a2_816b_71790bbb6727.slice/crio-6d98c3a6a009f72bebc4e10fb4bccb84bbf1510c86b77901fe181104f1e66abe WatchSource:0}: Error finding container 6d98c3a6a009f72bebc4e10fb4bccb84bbf1510c86b77901fe181104f1e66abe: Status 404 returned error can't find the container with id 6d98c3a6a009f72bebc4e10fb4bccb84bbf1510c86b77901fe181104f1e66abe
Nov 03 20:44:42.492342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:42.492310    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4" event={"ID":"4aa1413a-79c3-48a2-816b-71790bbb6727","Type":"ContainerStarted","Data":"6d98c3a6a009f72bebc4e10fb4bccb84bbf1510c86b77901fe181104f1e66abe"}
Nov 03 20:44:42.616853 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:42.616821    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:43.015768 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:43.015737    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:43.239490 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:43.239457    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:45.308837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:45.308798    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:45.692816 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:45.692744    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:44:45.693361 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:45.693343    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:44:45.706838 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:45.706818    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:44:45.707444 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:45.707423    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:44:46.783068 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:46.783032    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:47.034255 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.034176    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"]
Nov 03 20:44:47.038920 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.038901    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:47.084129 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.084106    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shmhc\" (UniqueName: \"kubernetes.io/projected/bd891d58-0ffd-4e01-a8d9-2710445a5880-kube-api-access-shmhc\") pod \"dns-operator-controller-manager-8557cbdb84-s7w5j\" (UID: \"bd891d58-0ffd-4e01-a8d9-2710445a5880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:47.154859 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.154826    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mcvss\""
Nov 03 20:44:47.154972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.154873    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 03 20:44:47.163663 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.163643    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"]
Nov 03 20:44:47.185239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.185218    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shmhc\" (UniqueName: \"kubernetes.io/projected/bd891d58-0ffd-4e01-a8d9-2710445a5880-kube-api-access-shmhc\") pod \"dns-operator-controller-manager-8557cbdb84-s7w5j\" (UID: \"bd891d58-0ffd-4e01-a8d9-2710445a5880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:47.266356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.266330    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shmhc\" (UniqueName: \"kubernetes.io/projected/bd891d58-0ffd-4e01-a8d9-2710445a5880-kube-api-access-shmhc\") pod \"dns-operator-controller-manager-8557cbdb84-s7w5j\" (UID: \"bd891d58-0ffd-4e01-a8d9-2710445a5880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:47.351098 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.351036    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:47.731006 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:47.730983    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"]
Nov 03 20:44:47.733201 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:47.733173    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd891d58_0ffd_4e01_a8d9_2710445a5880.slice/crio-47f4bef40c9e7d3d840f446223731e45c8c7074f6c6b17539affa1b6188622f0 WatchSource:0}: Error finding container 47f4bef40c9e7d3d840f446223731e45c8c7074f6c6b17539affa1b6188622f0: Status 404 returned error can't find the container with id 47f4bef40c9e7d3d840f446223731e45c8c7074f6c6b17539affa1b6188622f0
Nov 03 20:44:48.522635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:48.522599    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j" event={"ID":"bd891d58-0ffd-4e01-a8d9-2710445a5880","Type":"ContainerStarted","Data":"47f4bef40c9e7d3d840f446223731e45c8c7074f6c6b17539affa1b6188622f0"}
Nov 03 20:44:50.161179 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:50.161137    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:50.480580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:50.480526    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:50.747579 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:50.747442    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:50.892993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:50.892952    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:51.077753 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:51.077677    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:51.220204 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:51.220172    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:51.476954 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:51.476928    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:51.625817 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:51.625782    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:52.548803 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:52.548763    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j" event={"ID":"bd891d58-0ffd-4e01-a8d9-2710445a5880","Type":"ContainerStarted","Data":"c6ef5279bae4552157b3302f2edcec6afae3f8cc4f8fd16a06f889e14838e728"}
Nov 03 20:44:52.549262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:52.548846    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:44:52.607960 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:52.607930    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:52.669074 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:52.667444    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j" podStartSLOduration=2.686739079 podStartE2EDuration="6.66742438s" totalImagesPullingTime="3.980685301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:52.666826525 +0000 UTC m=+307.118195712" watchObservedRunningTime="2026-11-03 20:44:52.66742438 +0000 UTC m=+307.118793558"
Nov 03 20:44:52.765146 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:52.765110    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:53.179082 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:53.179046    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:53.555894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:53.555862    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4" event={"ID":"4aa1413a-79c3-48a2-816b-71790bbb6727","Type":"ContainerStarted","Data":"3869743d4ce71987bafd0623bd71e04a8ee2174f230d9e6644e86029557e32b0"}
Nov 03 20:44:53.556303 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:53.555953    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:44:53.674842 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:53.674779    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4" podStartSLOduration=1.606217198 podStartE2EDuration="12.674761842s" totalImagesPullingTime="11.068544644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:53.673910441 +0000 UTC m=+308.125279619" watchObservedRunningTime="2026-11-03 20:44:53.674761842 +0000 UTC m=+308.126131019"
Nov 03 20:44:54.812168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.812131    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:54.846344 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.846317    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"]
Nov 03 20:44:54.848464 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.848447    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:54.927398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.927369    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 03 20:44:54.949850 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.949825    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 03 20:44:54.950013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.949849    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wr9t8\""
Nov 03 20:44:54.951149 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.951130    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jph6\" (UniqueName: \"kubernetes.io/projected/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-kube-api-access-2jph6\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:54.951239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.951158    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tls-certs\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:54.951239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.951193    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tmp\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:54.951681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:54.951665    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"]
Nov 03 20:44:55.005958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.005934    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:55.052358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.052328    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jph6\" (UniqueName: \"kubernetes.io/projected/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-kube-api-access-2jph6\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.052514 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.052364    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tls-certs\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.052514 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.052391    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tmp\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.054894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.054865    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tmp\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.055027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.055010    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-tls-certs\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.113432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.113370    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jph6\" (UniqueName: \"kubernetes.io/projected/d7ac0f39-0dc9-403d-b4bb-a324a82c4958-kube-api-access-2jph6\") pod \"kube-auth-proxy-68d884f4c6-gwf8q\" (UID: \"d7ac0f39-0dc9-403d-b4bb-a324a82c4958\") " pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.159238 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.159209    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"
Nov 03 20:44:55.181464 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.181437    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:55.423700 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.423613    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:55.527799 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.527628    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q"]
Nov 03 20:44:55.529963 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:44:55.529939    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ac0f39_0dc9_403d_b4bb_a324a82c4958.slice/crio-9a8559d6257c18aab5dede7bf02a90490063709ac62c47da699ee0e4b29322d5 WatchSource:0}: Error finding container 9a8559d6257c18aab5dede7bf02a90490063709ac62c47da699ee0e4b29322d5: Status 404 returned error can't find the container with id 9a8559d6257c18aab5dede7bf02a90490063709ac62c47da699ee0e4b29322d5
Nov 03 20:44:55.566940 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.566911    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q" event={"ID":"d7ac0f39-0dc9-403d-b4bb-a324a82c4958","Type":"ContainerStarted","Data":"9a8559d6257c18aab5dede7bf02a90490063709ac62c47da699ee0e4b29322d5"}
Nov 03 20:44:55.568528 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:55.568506    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:58.309775 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:58.309715    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:58.472825 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:58.472791    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:44:58.586938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:58.586850    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q" event={"ID":"d7ac0f39-0dc9-403d-b4bb-a324a82c4958","Type":"ContainerStarted","Data":"fd9079d9504a54a4b41c023a5dc173c81b40497d8c276e8c81440d5f6d622494"}
Nov 03 20:44:58.704588 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:44:58.704519    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68d884f4c6-gwf8q" podStartSLOduration=1.9582662530000001 podStartE2EDuration="4.704505744s" totalImagesPullingTime="2.746239491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:58.702799757 +0000 UTC m=+313.154168934" watchObservedRunningTime="2026-11-03 20:44:58.704505744 +0000 UTC m=+313.155874921"
Nov 03 20:45:02.660946 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:02.660912    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:03.095629 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:03.095595    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:03.256213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:03.256177    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:03.558335 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:03.558306    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-s7w5j"
Nov 03 20:45:04.563504 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:04.563477    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-zxvm4"
Nov 03 20:45:05.555402 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:05.555369    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:05.705581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:05.705526    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:06.847837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:06.847803    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:06.989744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:06.989714    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:07.005706 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:45:07.005670    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:45:07.133370 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:07.133295    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:08.173518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:08.173491    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:11.584008 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.583975    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:11.659469 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.659437    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:11.662098 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.662083    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.714151 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.714125    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 03 20:45:11.714311 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.714276    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-77nnf\""
Nov 03 20:45:11.765547 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.765515    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:11.806749 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.806726    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.806892 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.806757    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhd5z\" (UniqueName: \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.903724 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.903648    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:11.907637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.907612    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.907754 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.907646    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhd5z\" (UniqueName: \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.908301 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.908280    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.948861 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.948814    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:11.968373 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.968340    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhd5z\" (UniqueName: \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\") pod \"limitador-limitador-74dfcfb894-8h6d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:11.972998 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:11.972977    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:12.104314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.104282    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:12.348933 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.348856    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:12.351507 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:45:12.351479    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod210e4543_ba3f_4192_9735_d6ca0a8f26d6.slice/crio-b7daf5e1be9b287ff11cc5e7e27c8a56fe57a7f936d8bfc0b22619045e7c6c4f WatchSource:0}: Error finding container b7daf5e1be9b287ff11cc5e7e27c8a56fe57a7f936d8bfc0b22619045e7c6c4f: Status 404 returned error can't find the container with id b7daf5e1be9b287ff11cc5e7e27c8a56fe57a7f936d8bfc0b22619045e7c6c4f
Nov 03 20:45:12.353608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.353591    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:45:12.509836 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.509807    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:12.665022 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.664942    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" event={"ID":"210e4543-ba3f-4192-9735-d6ca0a8f26d6","Type":"ContainerStarted","Data":"b7daf5e1be9b287ff11cc5e7e27c8a56fe57a7f936d8bfc0b22619045e7c6c4f"}
Nov 03 20:45:12.673315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.673281    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:12.908203 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:12.908174    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:13.055600 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:13.055570    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:13.197830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:13.197788    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:13.395806 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:13.395728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:13.542283 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:13.542246    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:14.678533 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:14.678440    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" event={"ID":"210e4543-ba3f-4192-9735-d6ca0a8f26d6","Type":"ContainerStarted","Data":"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"}
Nov 03 20:45:14.679049 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:14.678553    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:14.798901 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:14.798838    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" podStartSLOduration=1.941207657 podStartE2EDuration="3.798824144s" totalImagesPullingTime="1.857616487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:45:14.797381153 +0000 UTC m=+329.248750331" watchObservedRunningTime="2026-11-03 20:45:14.798824144 +0000 UTC m=+329.250193320"
Nov 03 20:45:19.727868 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:19.727832    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:19.875374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:19.875334    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:24.517807 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:24.517760    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:24.661760 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:24.661728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:25.684602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:25.684571    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:26.956549 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:26.956520    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:27.104745 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:27.104711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:31.775077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.775042    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:45:31.779538 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.779519    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:45:31.835946 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.835922    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:45:31.867935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.867910    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljpn\" (UniqueName: \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\") pod \"authorino-56c6c9ddd5-5cdkv\" (UID: \"14a31f1d-0252-4eb8-beba-9cce4932afc1\") " pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:45:31.882997 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.882976    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qg6jx\""
Nov 03 20:45:31.968699 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:31.968668    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qljpn\" (UniqueName: \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\") pod \"authorino-56c6c9ddd5-5cdkv\" (UID: \"14a31f1d-0252-4eb8-beba-9cce4932afc1\") " pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:45:32.028989 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:32.028922    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljpn\" (UniqueName: \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\") pod \"authorino-56c6c9ddd5-5cdkv\" (UID: \"14a31f1d-0252-4eb8-beba-9cce4932afc1\") " pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:45:32.090153 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:32.090125    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:45:32.466578 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:32.466540    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:45:32.468694 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:45:32.468666    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14a31f1d_0252_4eb8_beba_9cce4932afc1.slice/crio-90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7 WatchSource:0}: Error finding container 90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7: Status 404 returned error can't find the container with id 90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7
Nov 03 20:45:32.768864 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:32.768784    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" event={"ID":"14a31f1d-0252-4eb8-beba-9cce4932afc1","Type":"ContainerStarted","Data":"90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7"}
Nov 03 20:45:35.654816 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:35.654772    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:35.832494 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:35.832463    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:36.356243 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:36.356201    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:36.791631 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:36.791594    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" event={"ID":"14a31f1d-0252-4eb8-beba-9cce4932afc1","Type":"ContainerStarted","Data":"4741c3f721a98798335a373b645bd40c2407fde482bc238efe5fd58459a357c0"}
Nov 03 20:45:36.909329 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:36.909277    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" podStartSLOduration=2.083506621 podStartE2EDuration="5.909263538s" totalImagesPullingTime="3.825756917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:45:36.907925353 +0000 UTC m=+351.359294531" watchObservedRunningTime="2026-11-03 20:45:36.909263538 +0000 UTC m=+351.360632715"
Nov 03 20:45:37.002732 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:45:37.002695    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:45:46.609587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:46.609544    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:46.610079 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:46.609762    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" podUID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" containerName="limitador" containerID="cri-o://7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda" gracePeriod=30
Nov 03 20:45:47.163169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.163142    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:47.195427 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.195373    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\" (UniqueName: \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\") pod \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") "
Nov 03 20:45:47.195518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.195430    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\" (UniqueName: \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\") pod \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\" (UID: \"210e4543-ba3f-4192-9735-d6ca0a8f26d6\") "
Nov 03 20:45:47.195790 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.195764    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file" pod "210e4543-ba3f-4192-9735-d6ca0a8f26d6" (UID: "210e4543-ba3f-4192-9735-d6ca0a8f26d6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:45:47.197763 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.197739    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z" pod "210e4543-ba3f-4192-9735-d6ca0a8f26d6" (UID: "210e4543-ba3f-4192-9735-d6ca0a8f26d6"). InnerVolumeSpecName "kube-api-access-zhd5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:45:47.296499 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.296473    5915 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/210e4543-ba3f-4192-9735-d6ca0a8f26d6-config-file\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:45:47.296499 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.296497    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhd5z\" (UniqueName: \"kubernetes.io/projected/210e4543-ba3f-4192-9735-d6ca0a8f26d6-kube-api-access-zhd5z\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:45:47.850527 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.850493    5915 generic.go:356] "Generic (PLEG): container finished" podID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" containerID="7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda" exitCode=0
Nov 03 20:45:47.850924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.850577    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"
Nov 03 20:45:47.850924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.850583    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" event={"ID":"210e4543-ba3f-4192-9735-d6ca0a8f26d6","Type":"ContainerDied","Data":"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"}
Nov 03 20:45:47.850924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.850647    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-8h6d6" event={"ID":"210e4543-ba3f-4192-9735-d6ca0a8f26d6","Type":"ContainerDied","Data":"b7daf5e1be9b287ff11cc5e7e27c8a56fe57a7f936d8bfc0b22619045e7c6c4f"}
Nov 03 20:45:47.850924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.850677    5915 scope.go:122] "RemoveContainer" containerID="7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"
Nov 03 20:45:47.859577 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.859533    5915 scope.go:122] "RemoveContainer" containerID="7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"
Nov 03 20:45:47.859834 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:45:47.859817    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda\": container with ID starting with 7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda not found: ID does not exist" containerID="7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"
Nov 03 20:45:47.859882 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:47.859842    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda"} err="failed to get container status \"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda\": rpc error: code = NotFound desc = could not find container \"7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda\": container with ID starting with 7a3d7672fb7ad04da13d926b6a393c31bee1967f741a19bf6cc33f90ad66ceda not found: ID does not exist"
Nov 03 20:45:48.014125 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:48.014097    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:48.063273 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:48.063249    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-8h6d6"]
Nov 03 20:45:48.991844 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:48.991814    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:49.201744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:49.201711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:49.679892 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:49.679863    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" path="/var/lib/kubelet/pods/210e4543-ba3f-4192-9735-d6ca0a8f26d6/volumes"
Nov 03 20:45:57.995238 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:57.995149    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:58.076454 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.076419    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:45:58.076861 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.076846    5915 cpu_manager.go:414] "containerMap: removing container" podUID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" containerName="limitador"
Nov 03 20:45:58.076937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.076862    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" containerName="limitador"
Nov 03 20:45:58.076990 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.076971    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="210e4543-ba3f-4192-9735-d6ca0a8f26d6" containerName="limitador"
Nov 03 20:45:58.080042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.080025    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.133792 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.133766    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 20:45:58.133948 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.133795    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 03 20:45:58.133948 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.133800    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 03 20:45:58.150948 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.150919    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:58.180914 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.180887    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.181010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.180953    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.181053 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.181026    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrch\" (UniqueName: \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.183050 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.183033    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 20:45:58.183101 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.183036    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 03 20:45:58.183134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.183104    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tgsmh\""
Nov 03 20:45:58.185370 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.185351    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:45:58.281886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.281817    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.281886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.281861    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrch\" (UniqueName: \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.282069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.281895    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.284438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.284416    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.284568 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.284508    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.303941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.303918    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:58.342403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.342380    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrch\" (UniqueName: \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\") pod \"maas-controller-579cf7c6d-6ddnz\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") " pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.394405 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.394380    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:45:58.448120 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.448091    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:45:58.521945 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.521917    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 20:45:58.526391 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.526376    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.583863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.583837    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt4b7\" (UniqueName: \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.584010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.583899    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.584082 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.584006    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.586029 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.586004    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 20:45:58.685085 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.685054    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.685239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.685093    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.685239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.685162    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt4b7\" (UniqueName: \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.687618 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.687594    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.687715 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.687629    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.745382 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.745355    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt4b7\" (UniqueName: \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\") pod \"maas-controller-6778fc7d44-68lkl\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") " pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.767846 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.767812    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:45:58.770040 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:45:58.770018    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc54504f_bce4_4462_bc03_2037929d5284.slice/crio-29f74e04f7b059eaba9f58fcc4cd2ce7054574d8622d7ada7a05efa69f9673b5 WatchSource:0}: Error finding container 29f74e04f7b059eaba9f58fcc4cd2ce7054574d8622d7ada7a05efa69f9673b5: Status 404 returned error can't find the container with id 29f74e04f7b059eaba9f58fcc4cd2ce7054574d8622d7ada7a05efa69f9673b5
Nov 03 20:45:58.839364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.839306    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:45:58.912078 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:58.912047    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" event={"ID":"fc54504f-bce4-4462-bc03-2037929d5284","Type":"ContainerStarted","Data":"29f74e04f7b059eaba9f58fcc4cd2ce7054574d8622d7ada7a05efa69f9673b5"}
Nov 03 20:45:59.211976 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:59.211951    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 20:45:59.214021 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:45:59.213994    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccae0f1_0296_4f10_a808_03c9f2bad8c7.slice/crio-5e8a2a431968fe89c152d694dd672ffd35cbc70e961c83fadac8f2e7b76127d9 WatchSource:0}: Error finding container 5e8a2a431968fe89c152d694dd672ffd35cbc70e961c83fadac8f2e7b76127d9: Status 404 returned error can't find the container with id 5e8a2a431968fe89c152d694dd672ffd35cbc70e961c83fadac8f2e7b76127d9
Nov 03 20:45:59.921002 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:45:59.920962    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerStarted","Data":"5e8a2a431968fe89c152d694dd672ffd35cbc70e961c83fadac8f2e7b76127d9"}
Nov 03 20:46:02.546748 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.546718    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:02.712462 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.712426    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:02.941405 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.941312    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerStarted","Data":"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"}
Nov 03 20:46:02.941607 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.941439    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:46:02.942924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.942898    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" event={"ID":"fc54504f-bce4-4462-bc03-2037929d5284","Type":"ContainerStarted","Data":"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"}
Nov 03 20:46:02.943047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:02.942958    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:46:03.060164 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:03.060107    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6778fc7d44-68lkl" podStartSLOduration=2.141533232 podStartE2EDuration="5.060094825s" totalImagesPullingTime="2.918561593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:03.058539139 +0000 UTC m=+377.509908317" watchObservedRunningTime="2026-11-03 20:46:03.060094825 +0000 UTC m=+377.511464001"
Nov 03 20:46:03.176904 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:03.176848    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" podStartSLOduration=1.823685002 podStartE2EDuration="5.176834056s" totalImagesPullingTime="3.353149054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:03.175891731 +0000 UTC m=+377.627260900" watchObservedRunningTime="2026-11-03 20:46:03.176834056 +0000 UTC m=+377.628203243"
Nov 03 20:46:05.570531 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.570496    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"]
Nov 03 20:46:05.574116 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.574097    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.627043 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.627017    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 03 20:46:05.627181 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.627055    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 03 20:46:05.627356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.627340    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 03 20:46:05.647968 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.647943    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.648090 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.647977    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hqs\" (UniqueName: \"kubernetes.io/projected/623d2762-055f-48d9-b81b-28664f039e64-kube-api-access-67hqs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.648090 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.648008    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.648217 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.648192    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.676765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.676738    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-9kff7\""
Nov 03 20:46:05.680947 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.680923    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"]
Nov 03 20:46:05.717907 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.717865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:05.748908 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.748883    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.749045 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.748932    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.749045 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.748950    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hqs\" (UniqueName: \"kubernetes.io/projected/623d2762-055f-48d9-b81b-28664f039e64-kube-api-access-67hqs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.749168 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:05.749080    5915 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 03 20:46:05.749168 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:05.749156    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert podName:623d2762-055f-48d9-b81b-28664f039e64 nodeName:}" failed. No retries permitted until 2026-11-03 20:46:06.24913343 +0000 UTC m=+380.700502600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-zrrbs" (UID: "623d2762-055f-48d9-b81b-28664f039e64") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 03 20:46:05.749305 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.749181    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.749350 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:05.749317    5915 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 03 20:46:05.749389 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:05.749378    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source podName:623d2762-055f-48d9-b81b-28664f039e64 nodeName:}" failed. No retries permitted until 2026-11-03 20:46:06.249365639 +0000 UTC m=+380.700734812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-zrrbs" (UID: "623d2762-055f-48d9-b81b-28664f039e64") : configmap "usage-tenancy-proxy-source" not found
Nov 03 20:46:05.749697 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.749678    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.810403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.810375    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hqs\" (UniqueName: \"kubernetes.io/projected/623d2762-055f-48d9-b81b-28664f039e64-kube-api-access-67hqs\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:05.979795 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:05.979755    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:06.253587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.253502    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:06.253587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.253540    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:06.254358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.254283    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/623d2762-055f-48d9-b81b-28664f039e64-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:06.256071 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.256048    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/623d2762-055f-48d9-b81b-28664f039e64-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zrrbs\" (UID: \"623d2762-055f-48d9-b81b-28664f039e64\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:06.486337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.486299    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:06.859177 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.859151    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"]
Nov 03 20:46:06.862061 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:46:06.862027    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623d2762_055f_48d9_b81b_28664f039e64.slice/crio-29c5d0566dccab84e7441507b13e56d98bbbf426a3d8d1163ed2b6a923634f9f WatchSource:0}: Error finding container 29c5d0566dccab84e7441507b13e56d98bbbf426a3d8d1163ed2b6a923634f9f: Status 404 returned error can't find the container with id 29c5d0566dccab84e7441507b13e56d98bbbf426a3d8d1163ed2b6a923634f9f
Nov 03 20:46:06.965414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:06.965382    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs" event={"ID":"623d2762-055f-48d9-b81b-28664f039e64","Type":"ContainerStarted","Data":"29c5d0566dccab84e7441507b13e56d98bbbf426a3d8d1163ed2b6a923634f9f"}
Nov 03 20:46:07.000944 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:07.000910    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:46:08.522029 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:08.521979    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:13.950993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:13.950964    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:46:13.951398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:13.951060    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:46:14.223637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.223537    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:46:14.223791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.223758    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" podUID="fc54504f-bce4-4462-bc03-2037929d5284" containerName="manager" containerID="cri-o://ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a" gracePeriod=30
Nov 03 20:46:14.468668 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.468645    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:46:14.622984 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.622903    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\" (UniqueName: \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\") pod \"fc54504f-bce4-4462-bc03-2037929d5284\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") "
Nov 03 20:46:14.622984 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.622964    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\") pod \"fc54504f-bce4-4462-bc03-2037929d5284\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") "
Nov 03 20:46:14.623212 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.622989    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\") pod \"fc54504f-bce4-4462-bc03-2037929d5284\" (UID: \"fc54504f-bce4-4462-bc03-2037929d5284\") "
Nov 03 20:46:14.625322 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.625288    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert" pod "fc54504f-bce4-4462-bc03-2037929d5284" (UID: "fc54504f-bce4-4462-bc03-2037929d5284"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:14.625442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.625349    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch" pod "fc54504f-bce4-4462-bc03-2037929d5284" (UID: "fc54504f-bce4-4462-bc03-2037929d5284"). InnerVolumeSpecName "kube-api-access-bvrch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:46:14.625518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.625504    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs" pod "fc54504f-bce4-4462-bc03-2037929d5284" (UID: "fc54504f-bce4-4462-bc03-2037929d5284"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:14.723762 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.723728    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvrch\" (UniqueName: \"kubernetes.io/projected/fc54504f-bce4-4462-bc03-2037929d5284-kube-api-access-bvrch\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:14.723762 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.723758    5915 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-webhook-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:14.723951 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:14.723771    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc54504f-bce4-4462-bc03-2037929d5284-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:15.008175 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.008145    5915 generic.go:356] "Generic (PLEG): container finished" podID="fc54504f-bce4-4462-bc03-2037929d5284" containerID="ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a" exitCode=0
Nov 03 20:46:15.008659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.008191    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-6ddnz"
Nov 03 20:46:15.008659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.008240    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" event={"ID":"fc54504f-bce4-4462-bc03-2037929d5284","Type":"ContainerDied","Data":"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"}
Nov 03 20:46:15.008659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.008299    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-6ddnz" event={"ID":"fc54504f-bce4-4462-bc03-2037929d5284","Type":"ContainerDied","Data":"29f74e04f7b059eaba9f58fcc4cd2ce7054574d8622d7ada7a05efa69f9673b5"}
Nov 03 20:46:15.008659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.008333    5915 scope.go:122] "RemoveContainer" containerID="ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"
Nov 03 20:46:15.018746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.018729    5915 scope.go:122] "RemoveContainer" containerID="ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"
Nov 03 20:46:15.019019 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:15.019004    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a\": container with ID starting with ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a not found: ID does not exist" containerID="ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"
Nov 03 20:46:15.019064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.019030    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a"} err="failed to get container status \"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a\": rpc error: code = NotFound desc = could not find container \"ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a\": container with ID starting with ca2c6ae0ed2921c73329fa51f4e73e89cebbda61e928671e43bc9a908c26019a not found: ID does not exist"
Nov 03 20:46:15.200493 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.200464    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:46:15.203535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.203497    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-6ddnz"]
Nov 03 20:46:15.679890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:15.679857    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc54504f-bce4-4462-bc03-2037929d5284" path="/var/lib/kubelet/pods/fc54504f-bce4-4462-bc03-2037929d5284/volumes"
Nov 03 20:46:22.547710 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.547671    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"]
Nov 03 20:46:22.548189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.548109    5915 cpu_manager.go:414] "containerMap: removing container" podUID="fc54504f-bce4-4462-bc03-2037929d5284" containerName="manager"
Nov 03 20:46:22.548189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.548125    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc54504f-bce4-4462-bc03-2037929d5284" containerName="manager"
Nov 03 20:46:22.548269 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.548200    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc54504f-bce4-4462-bc03-2037929d5284" containerName="manager"
Nov 03 20:46:22.567770 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.567723    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602088 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602034    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602093    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nmc6\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-kube-api-access-7nmc6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602130    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602166    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602199    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602230    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb47871f-c00a-4e19-897d-3c89fa082b2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.602478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.602258    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.623614 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.623582    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 20:46:22.672627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.672599    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 20:46:22.672627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.672608    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 20:46:22.674499 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.673798    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cqrrn\""
Nov 03 20:46:22.674499 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.673845    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 20:46:22.675518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.675490    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"]
Nov 03 20:46:22.703013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.702976    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb47871f-c00a-4e19-897d-3c89fa082b2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703015    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703058    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703081    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nmc6\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-kube-api-access-7nmc6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703111    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703144    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.703227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.703176    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.704093 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.704067    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.706383 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.706357    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb47871f-c00a-4e19-897d-3c89fa082b2e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.706522 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.706500    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.706856 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.706837    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.707879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.707837    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.765176 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.765143    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.765332 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.765192    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nmc6\" (UniqueName: \"kubernetes.io/projected/cb47871f-c00a-4e19-897d-3c89fa082b2e-kube-api-access-7nmc6\") pod \"istiod-openshift-gateway-7ddd9cfdc8-8lj8l\" (UID: \"cb47871f-c00a-4e19-897d-3c89fa082b2e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:22.890021 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:22.889934    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:23.325194 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:23.324295    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"]
Nov 03 20:46:23.327066 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:46:23.327039    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb47871f_c00a_4e19_897d_3c89fa082b2e.slice/crio-d997b6910141bd4ae1d6bbcbf9a8149f67d2fa67e4398fe4c18e1a3f375fdd2d WatchSource:0}: Error finding container d997b6910141bd4ae1d6bbcbf9a8149f67d2fa67e4398fe4c18e1a3f375fdd2d: Status 404 returned error can't find the container with id d997b6910141bd4ae1d6bbcbf9a8149f67d2fa67e4398fe4c18e1a3f375fdd2d
Nov 03 20:46:23.955733 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:23.955698    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:24.072726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:24.072689    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l" event={"ID":"cb47871f-c00a-4e19-897d-3c89fa082b2e","Type":"ContainerStarted","Data":"d997b6910141bd4ae1d6bbcbf9a8149f67d2fa67e4398fe4c18e1a3f375fdd2d"}
Nov 03 20:46:24.106495 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:24.106427    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:27.090959 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.090926    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs" event={"ID":"623d2762-055f-48d9-b81b-28664f039e64","Type":"ContainerStarted","Data":"79327ce4a906aa45145deaa823933e0ac0b5ba18620b755cd649f46bc7b60cc6"}
Nov 03 20:46:27.091342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.090988    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:27.208532 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.208469    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs" podStartSLOduration=2.430847868 podStartE2EDuration="22.208450626s" totalImagesPullingTime="19.777602758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:27.208009466 +0000 UTC m=+401.659378642" watchObservedRunningTime="2026-11-03 20:46:27.208450626 +0000 UTC m=+401.659819804"
Nov 03 20:46:27.490164 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.490126    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:27.634867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.634838    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:27.777054 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.776978    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:27.923009 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:27.922966    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:28.065505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:28.065428    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:28.207896 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:28.207865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:28.356500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:28.356424    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:28.510218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:28.510183    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:28.656212 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:28.656130    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:29.763803 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:29.763768    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:31.842924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:31.842895    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:46:31.843221 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:31.842955    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:46:32.120597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.120489    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l" event={"ID":"cb47871f-c00a-4e19-897d-3c89fa082b2e","Type":"ContainerStarted","Data":"4c53eb6f795ad950d6780bc9e9788fba67216b6c1092c842c43edb8c216091f4"}
Nov 03 20:46:32.120597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.120579    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:32.122444 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.122418    5915 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-8lj8l container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:46:32.122572 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.122467    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l" podUID="cb47871f-c00a-4e19-897d-3c89fa082b2e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:46:32.467885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.467859    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:32.678721 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:32.678676    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:33.127426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:33.127388    5915 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-8lj8l container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:46:33.127828 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:33.127484    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l" podUID="cb47871f-c00a-4e19-897d-3c89fa082b2e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:46:34.590209 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:34.590180    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:34.788007 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:34.787973    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.002355 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.002319    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.163603 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.163575    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.358048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.357951    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.503342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.503299    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.652265 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.652191    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.825833 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.825802    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:35.969081 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:35.969051    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:36.113189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.113160    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:36.127842 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.127816    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l"
Nov 03 20:46:36.258912 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.258850    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:36.347541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.347493    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8lj8l" podStartSLOduration=5.834557107 podStartE2EDuration="14.347479183s" totalImagesPullingTime="8.512922076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:32.32553215 +0000 UTC m=+406.776901326" watchObservedRunningTime="2026-11-03 20:46:36.347479183 +0000 UTC m=+410.798848359"
Nov 03 20:46:36.403760 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.403708    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:36.548398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:36.548328    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:37.001844 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:46:37.001809    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:46:38.099901 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:38.099868    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zrrbs"
Nov 03 20:46:46.719174 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:46.719142    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:46.886322 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:46.886287    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:46:50.969519 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:50.969487    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:46:50.970065 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:50.969707    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" podUID="14a31f1d-0252-4eb8-beba-9cce4932afc1" containerName="authorino" containerID="cri-o://4741c3f721a98798335a373b645bd40c2407fde482bc238efe5fd58459a357c0" gracePeriod=30
Nov 03 20:46:51.221783 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.221700    5915 generic.go:356] "Generic (PLEG): container finished" podID="14a31f1d-0252-4eb8-beba-9cce4932afc1" containerID="4741c3f721a98798335a373b645bd40c2407fde482bc238efe5fd58459a357c0" exitCode=0
Nov 03 20:46:51.221903 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.221771    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" event={"ID":"14a31f1d-0252-4eb8-beba-9cce4932afc1","Type":"ContainerDied","Data":"4741c3f721a98798335a373b645bd40c2407fde482bc238efe5fd58459a357c0"}
Nov 03 20:46:51.221903 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.221823    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv" event={"ID":"14a31f1d-0252-4eb8-beba-9cce4932afc1","Type":"ContainerDied","Data":"90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7"}
Nov 03 20:46:51.221903 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.221842    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90e04f257b0751d1eef0abbbf467c0fa1d17781d58c4e57919534818aa496cb7"
Nov 03 20:46:51.227522 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.227496    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:46:51.364046 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.364018    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\" (UniqueName: \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\") pod \"14a31f1d-0252-4eb8-beba-9cce4932afc1\" (UID: \"14a31f1d-0252-4eb8-beba-9cce4932afc1\") "
Nov 03 20:46:51.366158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.366133    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn" pod "14a31f1d-0252-4eb8-beba-9cce4932afc1" (UID: "14a31f1d-0252-4eb8-beba-9cce4932afc1"). InnerVolumeSpecName "kube-api-access-qljpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:46:51.465093 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:51.465062    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qljpn\" (UniqueName: \"kubernetes.io/projected/14a31f1d-0252-4eb8-beba-9cce4932afc1-kube-api-access-qljpn\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:52.226625 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:52.226595    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5cdkv"
Nov 03 20:46:52.389698 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:52.389669    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:46:52.398485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:52.395218    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5cdkv"]
Nov 03 20:46:53.679194 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:46:53.679161    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14a31f1d-0252-4eb8-beba-9cce4932afc1" path="/var/lib/kubelet/pods/14a31f1d-0252-4eb8-beba-9cce4932afc1/volumes"
Nov 03 20:47:07.002093 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:47:07.002050    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:47:08.872502 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:47:08.872462    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:47:33.176513 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:47:33.176414    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:47:33.322614 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:47:33.322581    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:47:37.001406 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:47:37.001363    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:47:52.754342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:47:52.754306    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:47:52.913851 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:47:52.913817    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:07.002923 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:48:07.002884    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:48:09.223607 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:09.223553    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:28.446283 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:28.446252    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:28.593197 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:28.593154    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:30.728957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:30.728926    5915 generic.go:356] "Generic (PLEG): container finished" podID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerID="f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494" exitCode=1
Nov 03 20:48:30.729340 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:30.729012    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerDied","Data":"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"}
Nov 03 20:48:30.729340 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:30.729289    5915 scope.go:122] "RemoveContainer" containerID="f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"
Nov 03 20:48:31.736512 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:31.736484    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerStarted","Data":"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"}
Nov 03 20:48:31.736989 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:31.736797    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:48:37.003544 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:48:37.003506    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:48:42.744288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:42.744259    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 20:48:49.697717 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:49.697681    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:53.257487 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.257456    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-p6hxw"]
Nov 03 20:48:53.257885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.257844    5915 cpu_manager.go:414] "containerMap: removing container" podUID="14a31f1d-0252-4eb8-beba-9cce4932afc1" containerName="authorino"
Nov 03 20:48:53.257885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.257856    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14a31f1d-0252-4eb8-beba-9cce4932afc1" containerName="authorino"
Nov 03 20:48:53.257974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.257919    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="14a31f1d-0252-4eb8-beba-9cce4932afc1" containerName="authorino"
Nov 03 20:48:53.260923 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.260905    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.313257 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.313226    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 20:48:53.329479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.329450    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:53.361643 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361609    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdw5z\" (UniqueName: \"kubernetes.io/projected/4e78f464-2e1e-460b-8ea5-ade9414e73ea-kube-api-access-hdw5z\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.361780 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361650    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.361780 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361740    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.361780 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361773    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.361893 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361855    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.361937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.361899    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.362715 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.362697    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 20:48:53.362777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.362699    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wnljc\""
Nov 03 20:48:53.362777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.362721    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 03 20:48:53.364851 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.364835    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-p6hxw"]
Nov 03 20:48:53.463227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463192    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463238    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463302    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdw5z\" (UniqueName: \"kubernetes.io/projected/4e78f464-2e1e-460b-8ea5-ade9414e73ea-kube-api-access-hdw5z\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463328    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463369    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463745 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463724    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463792 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463733    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.463888 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.463861    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-home\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.465809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.465785    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e78f464-2e1e-460b-8ea5-ade9414e73ea-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.465988 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.465971    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e78f464-2e1e-460b-8ea5-ade9414e73ea-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.527800 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.527721    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdw5z\" (UniqueName: \"kubernetes.io/projected/4e78f464-2e1e-460b-8ea5-ade9414e73ea-kube-api-access-hdw5z\") pod \"echo-simulator-kserve-64b6f8f96c-p6hxw\" (UID: \"4e78f464-2e1e-460b-8ea5-ade9414e73ea\") " pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.574500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.574471    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:53.948875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:53.948820    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-p6hxw"]
Nov 03 20:48:53.951170 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:48:53.951136    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e78f464_2e1e_460b_8ea5_ade9414e73ea.slice/crio-534d5bd913450204eb8852e5f405ed662b5d8548c45946ca22dfdfde46e6c11e WatchSource:0}: Error finding container 534d5bd913450204eb8852e5f405ed662b5d8548c45946ca22dfdfde46e6c11e: Status 404 returned error can't find the container with id 534d5bd913450204eb8852e5f405ed662b5d8548c45946ca22dfdfde46e6c11e
Nov 03 20:48:54.856823 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:54.856793    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw" event={"ID":"4e78f464-2e1e-460b-8ea5-ade9414e73ea","Type":"ContainerStarted","Data":"534d5bd913450204eb8852e5f405ed662b5d8548c45946ca22dfdfde46e6c11e"}
Nov 03 20:48:56.329714 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.329670    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"]
Nov 03 20:48:56.333954 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.333927    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.388608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388580    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.388608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388611    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr4tx\" (UniqueName: \"kubernetes.io/projected/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-kube-api-access-wr4tx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.388849 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388714    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.388849 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388768    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.388849 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388832    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.389012 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.388887    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.426959 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.426927    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:48:56.435461 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.435440    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 03 20:48:56.437946 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.437925    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"]
Nov 03 20:48:56.489716 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489687    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.489821 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489720    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.489877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489854    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.489976 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489960    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.490026 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489982    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.490026 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.489996    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr4tx\" (UniqueName: \"kubernetes.io/projected/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-kube-api-access-wr4tx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.490119 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.490066    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.490119 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.490075    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.490229 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.490175    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.492423 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.492390    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.492500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.492456    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.551426 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.551389    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr4tx\" (UniqueName: \"kubernetes.io/projected/e2e2f602-9c96-44f1-9bce-b60c8c4abab4-kube-api-access-wr4tx\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m\" (UID: \"e2e2f602-9c96-44f1-9bce-b60c8c4abab4\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.649778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.649697    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:48:56.869206 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.869169    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw" event={"ID":"4e78f464-2e1e-460b-8ea5-ade9414e73ea","Type":"ContainerStarted","Data":"caf3623a00902ded262e18ff0752129660f4c9a2acf607e661f96079e177fb55"}
Nov 03 20:48:56.985347 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:56.985293    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw" podStartSLOduration=1.500400534 podStartE2EDuration="3.985279159s" totalImagesPullingTime="2.484878625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:48:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:48:56.985252279 +0000 UTC m=+551.436621490" watchObservedRunningTime="2026-11-03 20:48:56.985279159 +0000 UTC m=+551.436648336"
Nov 03 20:48:57.029843 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:48:57.029820    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e2f602_9c96_44f1_9bce_b60c8c4abab4.slice/crio-7f378d1409429ee5f7402f6e1546f44b856c779366d606363deea4a9cd258eaa WatchSource:0}: Error finding container 7f378d1409429ee5f7402f6e1546f44b856c779366d606363deea4a9cd258eaa: Status 404 returned error can't find the container with id 7f378d1409429ee5f7402f6e1546f44b856c779366d606363deea4a9cd258eaa
Nov 03 20:48:57.036791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:57.036768    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"]
Nov 03 20:48:57.575335 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:57.575307    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:48:57.879100 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:57.879016    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m" event={"ID":"e2e2f602-9c96-44f1-9bce-b60c8c4abab4","Type":"ContainerStarted","Data":"7f378d1409429ee5f7402f6e1546f44b856c779366d606363deea4a9cd258eaa"}
Nov 03 20:48:58.886631 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:58.886597    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m" event={"ID":"e2e2f602-9c96-44f1-9bce-b60c8c4abab4","Type":"ContainerStarted","Data":"b7cd19c00ff294fe3919de5ee936b2ad8e4f1e882d4067bd76c7055e24297ee9"}
Nov 03 20:48:59.002801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:59.002756    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m" podStartSLOduration=1.262466522 podStartE2EDuration="3.002742938s" totalImagesPullingTime="1.740276416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:48:59.001960903 +0000 UTC m=+553.453330080" watchObservedRunningTime="2026-11-03 20:48:59.002742938 +0000 UTC m=+553.454112109"
Nov 03 20:48:59.650456 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:48:59.650425    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:49:00.329816 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:00.329776    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:03.117263 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.117229    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"]
Nov 03 20:49:03.121064 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.121044    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153083 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153053    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153113    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153213 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153191    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbvc6\" (UniqueName: \"kubernetes.io/projected/999ea8f5-7307-4421-8caf-06d86bacc5a7-kube-api-access-lbvc6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153324 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153239    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153359 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153334    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.153399 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.153370    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999ea8f5-7307-4421-8caf-06d86bacc5a7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.209809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.209780    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:03.222374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.222351    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 03 20:49:03.224522 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.224502    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"]
Nov 03 20:49:03.254299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254276    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254420 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254303    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbvc6\" (UniqueName: \"kubernetes.io/projected/999ea8f5-7307-4421-8caf-06d86bacc5a7-kube-api-access-lbvc6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254420 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254322    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254445    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254476    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999ea8f5-7307-4421-8caf-06d86bacc5a7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254640    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254770 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254753    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254824 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254776    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.254915 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.254899    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.256933 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.256913    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/999ea8f5-7307-4421-8caf-06d86bacc5a7-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.257037 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.257019    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/999ea8f5-7307-4421-8caf-06d86bacc5a7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.315413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.315388    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbvc6\" (UniqueName: \"kubernetes.io/projected/999ea8f5-7307-4421-8caf-06d86bacc5a7-kube-api-access-lbvc6\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s\" (UID: \"999ea8f5-7307-4421-8caf-06d86bacc5a7\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.434136 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.434047    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:03.575661 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.575604    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:49:03.588178 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.588151    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:49:03.820042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.820020    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"]
Nov 03 20:49:03.822316 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:49:03.822294    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999ea8f5_7307_4421_8caf_06d86bacc5a7.slice/crio-56ac2e8bed698d182bad5cf5e8d07246a61e29f046d5237b1747daa2428be10a WatchSource:0}: Error finding container 56ac2e8bed698d182bad5cf5e8d07246a61e29f046d5237b1747daa2428be10a: Status 404 returned error can't find the container with id 56ac2e8bed698d182bad5cf5e8d07246a61e29f046d5237b1747daa2428be10a
Nov 03 20:49:03.914942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.914914    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s" event={"ID":"999ea8f5-7307-4421-8caf-06d86bacc5a7","Type":"ContainerStarted","Data":"56ac2e8bed698d182bad5cf5e8d07246a61e29f046d5237b1747daa2428be10a"}
Nov 03 20:49:03.926110 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:03.926082    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-p6hxw"
Nov 03 20:49:04.922819 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:04.922723    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s" event={"ID":"999ea8f5-7307-4421-8caf-06d86bacc5a7","Type":"ContainerStarted","Data":"108e6bf24704d539bda5dd9b7e7c9a425d3c908787ffbc15fcd84208252f18ba"}
Nov 03 20:49:05.041166 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:05.041106    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s" podStartSLOduration=1.313308409 podStartE2EDuration="2.041091811s" totalImagesPullingTime="727.783402ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:05.039487612 +0000 UTC m=+559.490856783" watchObservedRunningTime="2026-11-03 20:49:05.041091811 +0000 UTC m=+559.492460988"
Nov 03 20:49:05.435159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:05.435129    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:06.649859 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:06.649822    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:49:06.655701 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:06.655657    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:06.662191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:06.662172    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:49:06.944808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:06.944738    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m"
Nov 03 20:49:07.002708 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:49:07.002678    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:49:09.554974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:09.554939    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:10.920839 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:10.920789    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:11.066337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:11.066303    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:13.434531 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:13.434490    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:13.449251 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:13.449219    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:13.483264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:13.483226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:13.982168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:13.982137    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s"
Nov 03 20:49:17.183393 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:17.183361    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:29.144747 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:29.144710    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:31.354889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.354857    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:49:31.358459 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.358441    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.417454 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.417428    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mmsql\""
Nov 03 20:49:31.465405 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.465376    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:49:31.500623 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500584    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500654    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500742    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgckj\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500798    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500854    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500904    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.500958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500944    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.501175 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.500973    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.501175 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.501002    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602412    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602647    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgckj\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602705    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602783 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602756    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602819 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602803    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.602854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.602841    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603140    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603360 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603333    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603205    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603380    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603278    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603247    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603787 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603762    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.603837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.603771    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.605039 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.604979    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.605982 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.605956    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.664069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.664044    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.664323 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.664304    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgckj\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-277tg\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:31.672183 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:31.672158    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:32.104575 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:32.104536    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:49:32.106448 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:49:32.106422    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35145399_3608_4f72_aec4_6c27f0ebcbba.slice/crio-8f6a5b482211dee357372fc64bcf291282441f9aafaae2004adcb99aed190bd2 WatchSource:0}: Error finding container 8f6a5b482211dee357372fc64bcf291282441f9aafaae2004adcb99aed190bd2: Status 404 returned error can't find the container with id 8f6a5b482211dee357372fc64bcf291282441f9aafaae2004adcb99aed190bd2
Nov 03 20:49:33.074550 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:33.074515    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" event={"ID":"35145399-3608-4f72-aec4-6c27f0ebcbba","Type":"ContainerStarted","Data":"8f6a5b482211dee357372fc64bcf291282441f9aafaae2004adcb99aed190bd2"}
Nov 03 20:49:36.135853 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:36.135820    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:36.136246 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:36.135910    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:36.136246 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:36.135959    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:37.027299 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:49:37.027255    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:49:37.111584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:37.098842    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" event={"ID":"35145399-3608-4f72-aec4-6c27f0ebcbba","Type":"ContainerStarted","Data":"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"}
Nov 03 20:49:37.281877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:37.281786    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podStartSLOduration=2.255129913 podStartE2EDuration="6.281771749s" totalImagesPullingTime="4.026641836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:37.231620124 +0000 UTC m=+591.682989301" watchObservedRunningTime="2026-11-03 20:49:37.281771749 +0000 UTC m=+591.733140923"
Nov 03 20:49:37.673240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:37.673161    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:37.674894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:37.674864    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:37.675024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:37.674923    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:38.672774 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:38.672735    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:38.673170 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:38.672795    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:39.673225 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:39.673184    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:39.673658 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:39.673252    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:40.524879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:40.524843    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:40.673197 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:40.673164    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:40.673367 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:40.673229    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:41.673367 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:41.673329    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:41.673866 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:41.673418    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:42.672961 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:42.672926    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-277tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:49:42.673124 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:42.672985    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 03 20:49:44.384206 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:44.384175    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:44.384758 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:44.384579    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:44.385392 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:44.385374    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:49:45.518109 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.518077    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-78cdd7cfd6-c9vgv"]
Nov 03 20:49:45.521648 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.521630    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.559336 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.559304    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-789mv\" (UniqueName: \"kubernetes.io/projected/dddc4384-4ea1-43ae-917d-dec91c54d2e3-kube-api-access-789mv\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.559484 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.559467    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dddc4384-4ea1-43ae-917d-dec91c54d2e3-tls-cert\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.623351 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.623314    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 03 20:49:45.623503 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.623462    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qg6jx\""
Nov 03 20:49:45.626651 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.626628    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78cdd7cfd6-c9vgv"]
Nov 03 20:49:45.660211 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.660188    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dddc4384-4ea1-43ae-917d-dec91c54d2e3-tls-cert\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.660339 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.660224    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-789mv\" (UniqueName: \"kubernetes.io/projected/dddc4384-4ea1-43ae-917d-dec91c54d2e3-kube-api-access-789mv\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.662285 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.662261    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:45.680702 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.680679    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:49:45.699877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.699851    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.714414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.714389    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 03 20:49:45.724772 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.724748    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dddc4384-4ea1-43ae-917d-dec91c54d2e3-tls-cert\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.724978 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.724942    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-789mv\" (UniqueName: \"kubernetes.io/projected/dddc4384-4ea1-43ae-917d-dec91c54d2e3-kube-api-access-789mv\") pod \"authorino-78cdd7cfd6-c9vgv\" (UID: \"dddc4384-4ea1-43ae-917d-dec91c54d2e3\") " pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.731993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.731955    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:49:45.746897 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.746875    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:49:45.747225 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.747206    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:49:45.753865 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.753845    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-twgz9\""
Nov 03 20:49:45.755836 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.755809    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 03 20:49:45.760962 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.760936    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.761143 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.761123    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlhtv\" (UniqueName: \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.768719 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.768670    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:49:45.769666 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.769646    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:49:45.840879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.840856    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 03 20:49:45.862180 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.862157    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlhtv\" (UniqueName: \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.862304 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.862252    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.864723 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.864702    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:45.884933 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.884917    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qg6jx\""
Nov 03 20:49:45.893983 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.893965    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv"
Nov 03 20:49:45.928377 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:45.928348    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlhtv\" (UniqueName: \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\") pod \"ai-gateway-controller-55cd769764-pm7dd\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") " pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:46.017224 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.017186    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:46.051815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.051790    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:46.267171 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.267146    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-78cdd7cfd6-c9vgv"]
Nov 03 20:49:46.268486 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:49:46.268459    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddddc4384_4ea1_43ae_917d_dec91c54d2e3.slice/crio-854b398683e55af903b7788fb8783e89efe365960ba5a068854781374eb91b3c WatchSource:0}: Error finding container 854b398683e55af903b7788fb8783e89efe365960ba5a068854781374eb91b3c: Status 404 returned error can't find the container with id 854b398683e55af903b7788fb8783e89efe365960ba5a068854781374eb91b3c
Nov 03 20:49:46.391116 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.391096    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:49:46.393254 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:49:46.393221    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6ffabd_019f_4ed8_96d1_f33f49763e90.slice/crio-fb2f780b9f39d4949d9f40480d5c8edc4fdf352e2fa769b271503082a7bf7605 WatchSource:0}: Error finding container fb2f780b9f39d4949d9f40480d5c8edc4fdf352e2fa769b271503082a7bf7605: Status 404 returned error can't find the container with id fb2f780b9f39d4949d9f40480d5c8edc4fdf352e2fa769b271503082a7bf7605
Nov 03 20:49:46.494609 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.494580    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 20:49:46.499256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.499241    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.561204 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.561172    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 20:49:46.568963 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.568940    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569091 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569014    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569091 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569052    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569091 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569074    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569189    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569217    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569318 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569237    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569350    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.569435 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.569382    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8czp\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670430    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670464    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670495    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670522    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8czp\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670588    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670676    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670710    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670734    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670996 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670780    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.670996 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.670971    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.671315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.671287    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.671398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.671330    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.671398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.671354    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.671398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.671387    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.673197 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.673177    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.673275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.673256    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.731604 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.731547    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.731728 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.731716    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8czp\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-247zc\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:46.812314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:46.812288    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:47.180567 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.180516    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv" event={"ID":"dddc4384-4ea1-43ae-917d-dec91c54d2e3","Type":"ContainerStarted","Data":"854b398683e55af903b7788fb8783e89efe365960ba5a068854781374eb91b3c"}
Nov 03 20:49:47.181859 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.181834    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" event={"ID":"aa6ffabd-019f-4ed8-96d1-f33f49763e90","Type":"ContainerStarted","Data":"fb2f780b9f39d4949d9f40480d5c8edc4fdf352e2fa769b271503082a7bf7605"}
Nov 03 20:49:47.260325 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.260304    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 20:49:47.262310 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:49:47.262283    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f1cc95_8bb8_4edf_b784_ded87f169271.slice/crio-bb5c3523034801e2124165d2e94889e5d2f4532ad9a7ae5cc2e91ddd29b7b1df WatchSource:0}: Error finding container bb5c3523034801e2124165d2e94889e5d2f4532ad9a7ae5cc2e91ddd29b7b1df: Status 404 returned error can't find the container with id bb5c3523034801e2124165d2e94889e5d2f4532ad9a7ae5cc2e91ddd29b7b1df
Nov 03 20:49:47.264381 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.264349    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:47.264475 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.264425    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:47.264475 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:47.264468    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:48.190501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.190399    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv" event={"ID":"dddc4384-4ea1-43ae-917d-dec91c54d2e3","Type":"ContainerStarted","Data":"51cf30f93b96ce0c6442d6ff08bb77686d8b6d355d6a204e390408b5041779cd"}
Nov 03 20:49:48.192799 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.192754    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" event={"ID":"c3f1cc95-8bb8-4edf-b784-ded87f169271","Type":"ContainerStarted","Data":"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"}
Nov 03 20:49:48.192933 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.192878    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" event={"ID":"c3f1cc95-8bb8-4edf-b784-ded87f169271","Type":"ContainerStarted","Data":"bb5c3523034801e2124165d2e94889e5d2f4532ad9a7ae5cc2e91ddd29b7b1df"}
Nov 03 20:49:48.488739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.488675    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-78cdd7cfd6-c9vgv" podStartSLOduration=1.87806808 podStartE2EDuration="3.488659178s" totalImagesPullingTime="1.610591098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:48.347950895 +0000 UTC m=+602.799320074" watchObservedRunningTime="2026-11-03 20:49:48.488659178 +0000 UTC m=+602.940028355"
Nov 03 20:49:48.488924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.488772    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podStartSLOduration=2.488764981 podStartE2EDuration="2.488764981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:48.487990773 +0000 UTC m=+602.939359951" watchObservedRunningTime="2026-11-03 20:49:48.488764981 +0000 UTC m=+602.940134161"
Nov 03 20:49:48.822635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.818906    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:48.822635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.820339    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:48.822635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:48.820392    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:49.823346 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:49.822896    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:49.823346 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:49.822960    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:50.821582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:50.820942    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:50.821582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:50.821012    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:51.817376 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:51.816926    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:51.817376 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:51.816991    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:52.823731 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:52.823692    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:52.824232 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:52.823762    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:53.261047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:53.260998    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" event={"ID":"aa6ffabd-019f-4ed8-96d1-f33f49763e90","Type":"ContainerStarted","Data":"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"}
Nov 03 20:49:53.261246 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:53.261199    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:49:53.381144 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:53.381076    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" podStartSLOduration=1.899459443 podStartE2EDuration="8.381056653s" totalImagesPullingTime="6.48159721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:53.37872005 +0000 UTC m=+607.830089228" watchObservedRunningTime="2026-11-03 20:49:53.381056653 +0000 UTC m=+607.832425811"
Nov 03 20:49:53.813141 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:53.813102    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:53.813324 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:53.813167    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:54.816914 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:54.816655    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 03 20:49:54.816914 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:54.816716    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 03 20:49:55.817389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:55.817359    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:55.818004 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:55.817782    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:55.818442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:55.818424    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 20:49:56.884795 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:56.884757    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:49:57.032392 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:49:57.032357    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:04.268642 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:04.268606    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:50:07.002673 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:50:07.002635    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:50:09.916667 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:09.916636    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:16.746801 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.746773    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"]
Nov 03 20:50:16.754969 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.754948    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.898730 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898699    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.898883 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898774    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.898883 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898806    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.898961 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898910    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.898961 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898952    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq2bg\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-kube-api-access-bq2bg\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.899034 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.898989    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.899080 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.899065    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.899128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.899105    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.899198 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.899183    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:16.969963 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:16.969933    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"]
Nov 03 20:50:17.000157 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000093    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000157 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000130    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000173    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000200    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000223    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000270    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000304    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq2bg\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-kube-api-access-bq2bg\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000353    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000424    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000819 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000549    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.000953 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.000932    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.001088 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.001066    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.001191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.001094    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.001413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.001391    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.002822 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.002799    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.002928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.002909    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.061613 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.061586    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq2bg\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-kube-api-access-bq2bg\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.066459 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.066437    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb0d68f7-486e-44d7-a324-11f4ea590bb4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s\" (UID: \"fb0d68f7-486e-44d7-a324-11f4ea590bb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.068471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.068446    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:17.498933 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.498908    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"]
Nov 03 20:50:17.501341 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:50:17.501306    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb0d68f7_486e_44d7_a324_11f4ea590bb4.slice/crio-2486c22584c626778dcb2690205b8238d9c70ce98cd1590aa5993bd5471c3233 WatchSource:0}: Error finding container 2486c22584c626778dcb2690205b8238d9c70ce98cd1590aa5993bd5471c3233: Status 404 returned error can't find the container with id 2486c22584c626778dcb2690205b8238d9c70ce98cd1590aa5993bd5471c3233
Nov 03 20:50:17.503736 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.503701    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:17.503815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.503784    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:17.503854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:17.503821    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:18.431117 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:18.431084    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" event={"ID":"fb0d68f7-486e-44d7-a324-11f4ea590bb4","Type":"ContainerStarted","Data":"0d6301a244df90026b496a480e31c8c4747c97fe23957252521e9bc4665aa03b"}
Nov 03 20:50:18.431501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:18.431136    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" event={"ID":"fb0d68f7-486e-44d7-a324-11f4ea590bb4","Type":"ContainerStarted","Data":"2486c22584c626778dcb2690205b8238d9c70ce98cd1590aa5993bd5471c3233"}
Nov 03 20:50:18.622584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:18.614294    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podStartSLOduration=2.614260033 podStartE2EDuration="2.614260033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:50:18.611163474 +0000 UTC m=+633.062532652" watchObservedRunningTime="2026-11-03 20:50:18.614260033 +0000 UTC m=+633.065629211"
Nov 03 20:50:19.069184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:19.069150    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:19.070900 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:19.070874    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:19.071028 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:19.070923    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:19.968266 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:19.968234    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:50:19.968694 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:19.968529    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" podUID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" containerName="manager" containerID="cri-o://7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68" gracePeriod=30
Nov 03 20:50:20.069428 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.069397    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:20.069616 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.069461    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:20.256321 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.256298    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:50:20.342143 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.342107    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\") pod \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") "
Nov 03 20:50:20.342341 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.342169    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\" (UniqueName: \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\") pod \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\" (UID: \"aa6ffabd-019f-4ed8-96d1-f33f49763e90\") "
Nov 03 20:50:20.344367 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.344339    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv" pod "aa6ffabd-019f-4ed8-96d1-f33f49763e90" (UID: "aa6ffabd-019f-4ed8-96d1-f33f49763e90"). InnerVolumeSpecName "kube-api-access-mlhtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:50:20.344492 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.344426    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs" pod "aa6ffabd-019f-4ed8-96d1-f33f49763e90" (UID: "aa6ffabd-019f-4ed8-96d1-f33f49763e90"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:50:20.443077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443043    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlhtv\" (UniqueName: \"kubernetes.io/projected/aa6ffabd-019f-4ed8-96d1-f33f49763e90-kube-api-access-mlhtv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:50:20.443077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443074    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa6ffabd-019f-4ed8-96d1-f33f49763e90-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:50:20.443768 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443744    5915 generic.go:356] "Generic (PLEG): container finished" podID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" containerID="7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68" exitCode=0
Nov 03 20:50:20.443867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443788    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd"
Nov 03 20:50:20.443867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443830    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" event={"ID":"aa6ffabd-019f-4ed8-96d1-f33f49763e90","Type":"ContainerDied","Data":"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"}
Nov 03 20:50:20.443942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443887    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-55cd769764-pm7dd" event={"ID":"aa6ffabd-019f-4ed8-96d1-f33f49763e90","Type":"ContainerDied","Data":"fb2f780b9f39d4949d9f40480d5c8edc4fdf352e2fa769b271503082a7bf7605"}
Nov 03 20:50:20.443942 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.443915    5915 scope.go:122] "RemoveContainer" containerID="7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"
Nov 03 20:50:20.454100 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.454083    5915 scope.go:122] "RemoveContainer" containerID="7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"
Nov 03 20:50:20.454388 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:50:20.454351    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68\": container with ID starting with 7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68 not found: ID does not exist" containerID="7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"
Nov 03 20:50:20.454447 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.454396    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68"} err="failed to get container status \"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68\": rpc error: code = NotFound desc = could not find container \"7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68\": container with ID starting with 7d2da3f6f6035e321a3d916cdf42442f1529dc83a5427e538a00a70f7798ab68 not found: ID does not exist"
Nov 03 20:50:20.629917 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.629837    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:50:20.632639 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:20.632613    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-55cd769764-pm7dd"]
Nov 03 20:50:21.069576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:21.069520    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:21.069977 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:21.069614    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:21.591117 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:21.591078    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:21.680388 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:21.680354    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" path="/var/lib/kubelet/pods/aa6ffabd-019f-4ed8-96d1-f33f49763e90/volumes"
Nov 03 20:50:22.064929 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:22.064894    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:22.068980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:22.068959    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:22.069050 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:22.069006    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:23.069599 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:23.069549    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:23.070021 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:23.069633    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:23.584021 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:23.583983    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:23.734185 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:23.734155    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:24.069353 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:24.069319    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 03 20:50:24.069537 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:24.069374    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 03 20:50:25.200709 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:25.200616    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:25.366110 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:25.366070    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:50:25.755477 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:25.755445    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:25.755928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:25.755901    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:25.756820 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:50:25.756790    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s"
Nov 03 20:50:37.002681 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:50:37.002641    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:51:00.163924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:00.163882    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:00.316083 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:00.316052    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:00.805712 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:00.805679    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.014242 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.014211    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 20:51:01.014691 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.014676    5915 cpu_manager.go:414] "containerMap: removing container" podUID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" containerName="manager"
Nov 03 20:51:01.014691 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.014692    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" containerName="manager"
Nov 03 20:51:01.014807 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.014798    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa6ffabd-019f-4ed8-96d1-f33f49763e90" containerName="manager"
Nov 03 20:51:01.022591 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.019095    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029571 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029534    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029584    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6v9l\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029741 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029693    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029741 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029734    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029809 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029799    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029835    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.029879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.029856    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.030049 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.030032    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.030092 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.030059    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.084508 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.084433    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 20:51:01.098523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.098495    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-12a423-openshift-default-dockercfg-2ztvn\""
Nov 03 20:51:01.123392 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.123365    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.131264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131238    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131360 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131267    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131360 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131296    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131473 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131414    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6v9l\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131473 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131462    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131493    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131528    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131597    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131613    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131630    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.131994 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.131971    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.132163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.132010    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.132253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.132199    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.132253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.132227    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.134215 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.134195    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.134452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.134433    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.194463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.194394    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.194463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.194423    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6v9l\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.276676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.276646    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.334815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.334748    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:01.433864 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.433834    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.584130 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.584097    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.754784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.754751    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:01.777629 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.777605    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 20:51:01.779574 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:01.779527    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368f8fcf_b02e_4e3b_b5a9_20ca404d3f3b.slice/crio-f01ece02987ea7cb71d74b1a14fde76f92403eb3dec532e43f25d898feae3506 WatchSource:0}: Error finding container f01ece02987ea7cb71d74b1a14fde76f92403eb3dec532e43f25d898feae3506: Status 404 returned error can't find the container with id f01ece02987ea7cb71d74b1a14fde76f92403eb3dec532e43f25d898feae3506
Nov 03 20:51:01.781738 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.781699    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:01.781827 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.781765    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:01.781827 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.781794    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:01.904818 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:01.904788    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:02.074466 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.074390    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:02.225858 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.225820    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:02.335591 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.335504    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 20:51:02.339400 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.339380    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.392408 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.392367    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-862138-openshift-default-dockercfg-8dw8p\""
Nov 03 20:51:02.402776 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.402745    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 20:51:02.436944 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.436912    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:02.445345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445321    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445476 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445373    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445476 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445393    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445599 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445538    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445599 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445586    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445703 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445670    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95h9h\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445749 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445731    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445805 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445748    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.445864 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.445829    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547202 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547168    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547202 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547207    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547235    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547293    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547354    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547378    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547421    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547478 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547447    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547512    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95h9h\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547686    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547863    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.547910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.547883    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.548284 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.548197    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.548284 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.548292    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.550042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.550009    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.550315 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.550299    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.590739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.590678    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:02.607831 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.607807    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.608128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.608106    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95h9h\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.653236 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.653209    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:02.726540 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.726493    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerStarted","Data":"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"}
Nov 03 20:51:02.726762 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.726572    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerStarted","Data":"f01ece02987ea7cb71d74b1a14fde76f92403eb3dec532e43f25d898feae3506"}
Nov 03 20:51:02.871234 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.871137    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podStartSLOduration=2.871123731 podStartE2EDuration="2.871123731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:02.870338692 +0000 UTC m=+677.321707868" watchObservedRunningTime="2026-11-03 20:51:02.871123731 +0000 UTC m=+677.322492907"
Nov 03 20:51:02.882239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:02.882212    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:03.048910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.048873    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:03.116468 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.116450    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 20:51:03.118571 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:03.118534    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91eeaf6_da8d_41c0_8ff9_4f2f54dd380a.slice/crio-3a880913061f7f5a660a03d15cd85909fbbd89bba7742ab3f051c6b58b1e8c5c WatchSource:0}: Error finding container 3a880913061f7f5a660a03d15cd85909fbbd89bba7742ab3f051c6b58b1e8c5c: Status 404 returned error can't find the container with id 3a880913061f7f5a660a03d15cd85909fbbd89bba7742ab3f051c6b58b1e8c5c
Nov 03 20:51:03.120855 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.120820    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:03.120964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.120899    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:03.120964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.120942    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:03.335585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.335509    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:03.346452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.346417    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:03.736137 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.736104    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerStarted","Data":"f74adf22cccdd264d4c1eab4dac873779a69c2e4ea7d301e03d45a871c5817da"}
Nov 03 20:51:03.736316 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.736180    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerStarted","Data":"3a880913061f7f5a660a03d15cd85909fbbd89bba7742ab3f051c6b58b1e8c5c"}
Nov 03 20:51:03.736371 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.736319    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:03.737243 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.737221    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:03.861429 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:03.861380    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podStartSLOduration=1.861360769 podStartE2EDuration="1.861360769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:03.859283851 +0000 UTC m=+678.310653029" watchObservedRunningTime="2026-11-03 20:51:03.861360769 +0000 UTC m=+678.312729946"
Nov 03 20:51:04.653815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:04.653782    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:04.658994 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:04.658972    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:04.743729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:04.743689    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:04.744414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:04.744393    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:51:05.465837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:05.465801    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:05.653219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:05.653190    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:05.935127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:05.935018    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:06.287841 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:06.287811    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:06.505816 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:06.505770    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:06.657720 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:06.657621    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:06.921248 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:06.921156    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:07.003725 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:51:07.003690    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:51:07.147623 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:07.147589    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:07.332163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:07.332132    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:07.747189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:07.747152    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:07.943181 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:07.943148    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:08.103417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:08.103329    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:08.379594 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:08.379461    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:08.725703 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:08.725673    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:08.980513 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:08.980439    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:10.120347 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:10.120315    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:10.281799 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:10.281750    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:10.462852 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:10.462820    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:10.754935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:10.754846    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:10.925737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:10.925706    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:11.341028 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.340993    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 20:51:11.344935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.344917    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.397207 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.397180    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-ca4642\""
Nov 03 20:51:11.405596 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.405537    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 20:51:11.443115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.443086    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-ca4642-dockercfg-9vzjm\""
Nov 03 20:51:11.455895 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.455864    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cq4\" (UniqueName: \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.456057 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.456037    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.557016 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.556977    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cq4\" (UniqueName: \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.557172 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.557080    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.557874 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.557853    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.617479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.617405    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cq4\" (UniqueName: \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\") pod \"payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.659937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.659900    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:11.974055 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:11.974015    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:12.051700 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:12.051656    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 20:51:12.056095 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:12.056073    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:51:12.122104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:12.122075    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:12.301226 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:12.301154    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:12.798580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:12.798537    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" event={"ID":"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d","Type":"ContainerStarted","Data":"adebfeb2e75da4a6501b4a2c6cc333bd3a955c429c918340fb9d498bbb3490cd"}
Nov 03 20:51:13.040735 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:13.040699    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:13.255851 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:13.255825    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:13.408775 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:13.408728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:15.786406 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:15.786372    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:15.817685 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:15.817649    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" event={"ID":"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d","Type":"ContainerStarted","Data":"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"}
Nov 03 20:51:15.817865 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:15.817764    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:15.951037 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:15.950971    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" podStartSLOduration=2.191053576 podStartE2EDuration="4.950949776s" totalImagesPullingTime="2.7598962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:15.938657831 +0000 UTC m=+690.390027008" watchObservedRunningTime="2026-11-03 20:51:15.950949776 +0000 UTC m=+690.402318953"
Nov 03 20:51:16.000283 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:16.000232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:17.315132 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:17.315084    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:17.470788 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:17.470754    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:17.628994 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:17.628880    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:18.394169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.394134    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:18.580814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.580781    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:18.816857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.816821    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 20:51:18.823461 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.823429    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:18.928255 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.928223    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-89bb58\""
Nov 03 20:51:18.933726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.929822    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-89bb58-dockercfg-sccvc\""
Nov 03 20:51:18.938302 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.938277    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 20:51:18.950793 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.946301    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:18.950793 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.946405    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6tf9\" (UniqueName: \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:18.980294 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.980264    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 20:51:18.984081 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:18.984061    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.029538 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.029509    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 20:51:19.051679 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.051644    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.051824 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.051717    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6tf9\" (UniqueName: \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.051824 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.051780    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.051941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.051827    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h85xf\" (UniqueName: \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.052389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.052369    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.112504 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.112437    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6tf9\" (UniqueName: \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\") pod \"payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.137450 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.137423    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.152596 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.152549    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.152693 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.152619    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h85xf\" (UniqueName: \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.153256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.153236    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.213636 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.213610    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h85xf\" (UniqueName: \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\") pod \"payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.297387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.297356    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.514060 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.514033    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 20:51:19.516244 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:19.516221    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36495e2b_373e_4985_b0e7_fb0b397352b9.slice/crio-5b37461c2935306523c0fd818c9854219847203f41016a4e701ec61041dbc9cd WatchSource:0}: Error finding container 5b37461c2935306523c0fd818c9854219847203f41016a4e701ec61041dbc9cd: Status 404 returned error can't find the container with id 5b37461c2935306523c0fd818c9854219847203f41016a4e701ec61041dbc9cd
Nov 03 20:51:19.678127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.672643    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 20:51:19.846184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.846092    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" event={"ID":"0fccc01f-9b8b-42d7-8c81-765059c9f791","Type":"ContainerStarted","Data":"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"}
Nov 03 20:51:19.846184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.846144    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" event={"ID":"0fccc01f-9b8b-42d7-8c81-765059c9f791","Type":"ContainerStarted","Data":"62995504ae6b196b987e79bfde21438f4ad4844bd9b77919d7597a6883cc0ce4"}
Nov 03 20:51:19.846184 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.846173    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:19.847855 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.847832    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" event={"ID":"36495e2b-373e-4985-b0e7-fb0b397352b9","Type":"ContainerStarted","Data":"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"}
Nov 03 20:51:19.847976 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.847870    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" event={"ID":"36495e2b-373e-4985-b0e7-fb0b397352b9","Type":"ContainerStarted","Data":"5b37461c2935306523c0fd818c9854219847203f41016a4e701ec61041dbc9cd"}
Nov 03 20:51:19.847976 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.847925    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:19.963200 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:19.963150    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" podStartSLOduration=1.9631370559999999 podStartE2EDuration="1.963137056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:19.961256622 +0000 UTC m=+694.412625801" watchObservedRunningTime="2026-11-03 20:51:19.963137056 +0000 UTC m=+694.414506233"
Nov 03 20:51:20.078385 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:20.078328    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" podStartSLOduration=2.078314628 podStartE2EDuration="2.078314628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:20.076136799 +0000 UTC m=+694.527505974" watchObservedRunningTime="2026-11-03 20:51:20.078314628 +0000 UTC m=+694.529683805"
Nov 03 20:51:20.938147 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:20.938114    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:21.141375 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.141345    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:21.525971 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.525931    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 20:51:21.536840 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.536812    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.588843 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.588811    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 20:51:21.589337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.589310    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.589511 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.589487    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmfm\" (UniqueName: \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.635736 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.635705    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-499187-dockercfg-2zm8r\""
Nov 03 20:51:21.640327 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.640298    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-499187\""
Nov 03 20:51:21.691042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.691007    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.691219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.691088    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmfm\" (UniqueName: \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.691723 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.691703    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.752168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.752140    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmfm\" (UniqueName: \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\") pod \"payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:21.851883 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:21.851798    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:22.248740 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.248711    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 20:51:22.250097 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:22.250058    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea5b727_d740_4455_8b7d_951460fe3443.slice/crio-6b4c351ace053dd9a28421485a9a018bc053d19a43b9d5e56131d3ba9fced1d8 WatchSource:0}: Error finding container 6b4c351ace053dd9a28421485a9a018bc053d19a43b9d5e56131d3ba9fced1d8: Status 404 returned error can't find the container with id 6b4c351ace053dd9a28421485a9a018bc053d19a43b9d5e56131d3ba9fced1d8
Nov 03 20:51:22.694414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.694333    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:22.866505 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.866474    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" event={"ID":"1ea5b727-d740-4455-8b7d-951460fe3443","Type":"ContainerStarted","Data":"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"}
Nov 03 20:51:22.866706 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.866524    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" event={"ID":"1ea5b727-d740-4455-8b7d-951460fe3443","Type":"ContainerStarted","Data":"6b4c351ace053dd9a28421485a9a018bc053d19a43b9d5e56131d3ba9fced1d8"}
Nov 03 20:51:22.866706 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.866607    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:22.889163 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.889129    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:22.983612 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:22.983531    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" podStartSLOduration=1.983516598 podStartE2EDuration="1.983516598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:22.98221288 +0000 UTC m=+697.433582076" watchObservedRunningTime="2026-11-03 20:51:22.983516598 +0000 UTC m=+697.434885775"
Nov 03 20:51:23.032794 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:23.032756    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:25.883186 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:25.883156    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:26.068792 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.068763    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:26.314131 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.314081    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 20:51:26.324897 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.324853    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.369378 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.369351    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 20:51:26.416843 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.416815    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-e22abf\""
Nov 03 20:51:26.417230 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.417214    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-e22abf-dockercfg-nxg9f\""
Nov 03 20:51:26.476147 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.476113    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.476328 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.476231    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl6hd\" (UniqueName: \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.577374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.577267    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.577374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.577342    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl6hd\" (UniqueName: \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.578139 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.578114    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.637978 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.637955    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl6hd\" (UniqueName: \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\") pod \"payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.638147 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.638131    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:26.828200 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:26.828129    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 20:51:27.056864 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.056835    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 20:51:27.058762 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:27.058735    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f1600e_7212_4d38_bc34_85ee90efebad.slice/crio-240d765d0ada05df121e9f1815818e90ce9bb266950b9a5b2dc9d6c965c3c45d WatchSource:0}: Error finding container 240d765d0ada05df121e9f1815818e90ce9bb266950b9a5b2dc9d6c965c3c45d: Status 404 returned error can't find the container with id 240d765d0ada05df121e9f1815818e90ce9bb266950b9a5b2dc9d6c965c3c45d
Nov 03 20:51:27.516673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.516633    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:27.670310 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.670273    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:27.906747 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.906674    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" event={"ID":"48f1600e-7212-4d38-bc34-85ee90efebad","Type":"ContainerStarted","Data":"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"}
Nov 03 20:51:27.906747 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.906730    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" event={"ID":"48f1600e-7212-4d38-bc34-85ee90efebad","Type":"ContainerStarted","Data":"240d765d0ada05df121e9f1815818e90ce9bb266950b9a5b2dc9d6c965c3c45d"}
Nov 03 20:51:27.906998 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.906773    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:27.982957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:27.982929    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:28.023429 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:28.023362    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" podStartSLOduration=2.023341606 podStartE2EDuration="2.023341606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:28.021385127 +0000 UTC m=+702.472754304" watchObservedRunningTime="2026-11-03 20:51:28.023341606 +0000 UTC m=+702.474710874"
Nov 03 20:51:28.144639 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:28.144606    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:28.294983 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:28.294947    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:30.856896 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.856866    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 20:51:30.857361 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.856943    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 20:51:30.894589 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.894540    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:30.930583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.930537    5915 generic.go:356] "Generic (PLEG): container finished" podID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerID="70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5" exitCode=0
Nov 03 20:51:30.930711 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.930612    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerDied","Data":"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"}
Nov 03 20:51:30.930936 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.930923    5915 scope.go:122] "RemoveContainer" containerID="70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"
Nov 03 20:51:30.932257 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.932234    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:30.932313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.932300    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:30.932348 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:30.932332    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:31.075043 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:31.074997    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:31.939714 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:31.939679    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerStarted","Data":"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"}
Nov 03 20:51:32.335854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:32.335815    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:32.337105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:32.337079    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 03 20:51:32.337180 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:32.337135    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 03 20:51:33.024046 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.024006    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 20:51:33.029122 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.029098    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.080674 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.080644    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.080818 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.080698    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc22t\" (UniqueName: \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.080910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.080850    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.080910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.080884    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.081928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.081907    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 20:51:33.127569 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.127519    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-e22abf-serving-cert\""
Nov 03 20:51:33.127739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.127612    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lnz6c\""
Nov 03 20:51:33.127739 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.127680    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 20:51:33.130649 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.130622    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 20:51:33.130649 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.130627    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 20:51:33.181932 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.181903    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc22t\" (UniqueName: \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.182099 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.181984    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.182099 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.182000    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.182099 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.182078    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.182255 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:51:33.182123    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-serving-cert: secret "maas-api-e2e-worker-w5-e22abf-serving-cert" not found
Nov 03 20:51:33.182255 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:51:33.182197    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls podName:89775bcb-6ffe-4af7-812e-1e7141bbca59 nodeName:}" failed. No retries permitted until 2026-11-03 20:51:33.682177941 +0000 UTC m=+708.133547101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls") pod "maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" (UID: "89775bcb-6ffe-4af7-812e-1e7141bbca59") : secret "maas-api-e2e-worker-w5-e22abf-serving-cert" not found
Nov 03 20:51:33.182798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.182778    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.184650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.184629    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.223610 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.221297    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:33.244415 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.244388    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc22t\" (UniqueName: \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.336194 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.336097    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 03 20:51:33.336333 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.336191    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 03 20:51:33.369905 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.369867    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:33.520596 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.520460    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:33.687833 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.687749    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.690366 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.690338    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:33.736541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.736514    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:33.875661 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.875632    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 20:51:33.942787 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:33.942711    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:34.335661 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:34.335628    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 03 20:51:34.336061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:34.335675    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 03 20:51:34.389384 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:34.389359    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 20:51:34.390764 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:34.390721    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89775bcb_6ffe_4af7_812e_1e7141bbca59.slice/crio-32ee4c1b5c9d17f58882cb4272a4a93ed51ef619902ccd11dd35d0b90390e60e WatchSource:0}: Error finding container 32ee4c1b5c9d17f58882cb4272a4a93ed51ef619902ccd11dd35d0b90390e60e: Status 404 returned error can't find the container with id 32ee4c1b5c9d17f58882cb4272a4a93ed51ef619902ccd11dd35d0b90390e60e
Nov 03 20:51:34.965617 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:34.965583    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" event={"ID":"89775bcb-6ffe-4af7-812e-1e7141bbca59","Type":"ContainerStarted","Data":"32ee4c1b5c9d17f58882cb4272a4a93ed51ef619902ccd11dd35d0b90390e60e"}
Nov 03 20:51:35.335729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:35.335654    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 03 20:51:35.335729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:35.335715    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 03 20:51:36.125597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.124941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:36.325193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.325148    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:36.335252 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.335216    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 03 20:51:36.335391 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.335274    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 03 20:51:36.982220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.982079    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" event={"ID":"89775bcb-6ffe-4af7-812e-1e7141bbca59","Type":"ContainerStarted","Data":"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"}
Nov 03 20:51:36.982220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:36.982179    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:37.025811 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:51:37.025775    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:51:37.131593 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:37.131470    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" podStartSLOduration=2.649237013 podStartE2EDuration="5.131455652s" totalImagesPullingTime="2.482218639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:37.128376356 +0000 UTC m=+711.579745639" watchObservedRunningTime="2026-11-03 20:51:37.131455652 +0000 UTC m=+711.582824828"
Nov 03 20:51:37.340054 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:37.340025    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:37.341068 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:37.341046    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 20:51:38.178192 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:38.178158    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:38.337979 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:38.337951    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:38.496680 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:38.496644    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:38.917406 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:38.917328    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 20:51:42.993648 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:42.993616    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 20:51:43.212489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:43.212452    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:45.783847 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:45.783818    5915 scope.go:122] "RemoveContainer" containerID="4741c3f721a98798335a373b645bd40c2407fde482bc238efe5fd58459a357c0"
Nov 03 20:51:48.017510 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:48.017472    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:52.923968 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:52.923935    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:56.261937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:56.261856    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:56.944230 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:56.944191    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:57.108200 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.108153    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:57.741196 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.741159    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 20:51:57.745207 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.745186    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:57.797737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.797711    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-862138-serving-cert\""
Nov 03 20:51:57.850166 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.850132    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 20:51:57.919143 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.919107    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:57.919336 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.919187    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:57.919336 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.919265    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvg4x\" (UniqueName: \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:57.919336 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.919312    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:57.972057 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:57.972020    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:58.021068 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.020956    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.021068 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.021020    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.021326 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.021074    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvg4x\" (UniqueName: \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.021326 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.021110    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.021875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.021845    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.024753 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.024730    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.024869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.024783    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.083921 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.083890    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvg4x\" (UniqueName: \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\") pod \"maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.172536 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.172497    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:51:58.358758 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.358672    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:51:58.735805 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:58.735776    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 20:51:58.737876 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:51:58.737849    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod512d6e6d_1e83_4c71_8715_ee44734cd867.slice/crio-a70fd7ad163e84cc3f189c80184e4204194effbc84396da82b11825b69dfce56 WatchSource:0}: Error finding container a70fd7ad163e84cc3f189c80184e4204194effbc84396da82b11825b69dfce56: Status 404 returned error can't find the container with id a70fd7ad163e84cc3f189c80184e4204194effbc84396da82b11825b69dfce56
Nov 03 20:51:59.122668 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:51:59.122584    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" event={"ID":"512d6e6d-1e83-4c71-8715-ee44734cd867","Type":"ContainerStarted","Data":"a70fd7ad163e84cc3f189c80184e4204194effbc84396da82b11825b69dfce56"}
Nov 03 20:52:00.130395 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:00.130355    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" event={"ID":"512d6e6d-1e83-4c71-8715-ee44734cd867","Type":"ContainerStarted","Data":"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"}
Nov 03 20:52:00.130885 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:00.130445    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:52:00.249671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:00.249614    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" podStartSLOduration=2.403824223 podStartE2EDuration="3.24959976s" totalImagesPullingTime="845.775537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:52:00.247209874 +0000 UTC m=+734.698579054" watchObservedRunningTime="2026-11-03 20:52:00.24959976 +0000 UTC m=+734.700968938"
Nov 03 20:52:00.799675 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:00.799629    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:01.458815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:01.458779    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:02.148417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:02.148385    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:02.339910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:02.339876    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:03.517180 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:03.517153    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:06.142179 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:06.142151    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 20:52:07.027743 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:52:07.027701    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:52:09.079550 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:09.079519    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:10.724332 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:10.724284    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:13.279807 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:13.279774    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:13.942913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:13.942873    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:14.166224 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:14.166193    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:14.507193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:14.507160    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:15.265338 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.265256    5915 generic.go:356] "Generic (PLEG): container finished" podID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerID="f74adf22cccdd264d4c1eab4dac873779a69c2e4ea7d301e03d45a871c5817da" exitCode=0
Nov 03 20:52:15.265593 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.265339    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerDied","Data":"f74adf22cccdd264d4c1eab4dac873779a69c2e4ea7d301e03d45a871c5817da"}
Nov 03 20:52:15.265757 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.265743    5915 scope.go:122] "RemoveContainer" containerID="f74adf22cccdd264d4c1eab4dac873779a69c2e4ea7d301e03d45a871c5817da"
Nov 03 20:52:15.267072 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.267049    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:52:15.267123 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.267113    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:52:15.267158 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:15.267143    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:52:16.272898 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:16.272861    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerStarted","Data":"6718d7a89115ac2811e62f6a5853632eba462c3eadb19532f4c6226fb4d66ea4"}
Nov 03 20:52:16.654293 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:16.654215    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:52:16.655463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:16.655442    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 03 20:52:16.655595 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:16.655493    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 03 20:52:17.445679 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:17.445644    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:17.653825 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:17.653787    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 03 20:52:17.654001 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:17.653843    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 03 20:52:18.231199 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:18.231169    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:18.530053 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:18.529972    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:18.653537 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:18.653501    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 03 20:52:18.653727 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:18.653551    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 03 20:52:19.654436 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:19.654405    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 03 20:52:19.654854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:19.654455    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 03 20:52:19.743584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:19.743529    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:52:20.176624 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:20.176584    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:20.654443 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:20.654413    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 03 20:52:20.654878 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:20.654478    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 03 20:52:21.665862 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:21.665828    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:52:21.667414 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:21.667390    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 20:52:22.706452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:22.706410    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:23.372531 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:23.372495    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:23.545879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:23.545846    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:25.576695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:25.576661    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:26.790018 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:26.789987    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:27.442262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:27.442227    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:27.650461 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:27.650418    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:30.659483 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:30.659448    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:30.815061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:30.815032    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:32.927250 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:32.927222    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:34.067925 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:34.067893    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:35.617499 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:35.617464    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:37.004121 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:52:37.004086    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:52:40.643329 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:40.643295    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:45.427259 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:45.427227    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:47.702501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:47.702468    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:48.529277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:48.529246    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:49.124814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:49.124782    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:49.297745 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:49.297681    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:50.467284 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:50.467249    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:55.325826 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:55.325795    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:52:55.835091 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:52:55.835054    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:00.243589 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:00.243529    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:05.159154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:05.159110    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:05.932254 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:05.932221    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:07.003752 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:53:07.003714    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:53:09.258033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:09.258000    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:10.055406 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:10.055370    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:10.978013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:10.977973    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:14.940285 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:14.940244    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:15.629300 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:15.629251    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:19.293301 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.293271    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 20:53:19.297128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.297109    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.350342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.350309    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"simulator-w2-862138-kserve-self-signed-certs\""
Nov 03 20:53:19.377191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377160    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.377304 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377225    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.377348 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377323    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6k69\" (UniqueName: \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.377472 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377452    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.377522 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377507    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.377581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.377537    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.385016 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.384991    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:19.399724 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.399707    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"default-dockercfg-fsbwz\""
Nov 03 20:53:19.400116 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.400099    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"kube-root-ca.crt\""
Nov 03 20:53:19.400201 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.400145    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"openshift-service-ca.crt\""
Nov 03 20:53:19.402564 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.402531    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 20:53:19.478190 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478158    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478190 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478191    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478222    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478260    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478281    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6k69\" (UniqueName: \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478366    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478645    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478734 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478696    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.478772 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.478752    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.480690 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.480670    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.480983 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.480964    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.539344 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.539319    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6k69\" (UniqueName: \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\") pod \"simulator-w2-862138-kserve-5bfbf44957-zthkt\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") " pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.632990 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.632900    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:19.834265 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:19.834232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:20.026695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:20.026668    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 20:53:20.028875 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:53:20.028850    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9fee32f_9d31_455d_8b37_c302bf946472.slice/crio-bf4592ee46d15863accbf36c0ad65b62fe0cca1b26a4b512aba654c556611a5c WatchSource:0}: Error finding container bf4592ee46d15863accbf36c0ad65b62fe0cca1b26a4b512aba654c556611a5c: Status 404 returned error can't find the container with id bf4592ee46d15863accbf36c0ad65b62fe0cca1b26a4b512aba654c556611a5c
Nov 03 20:53:20.702690 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:20.702660    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" event={"ID":"b9fee32f-9d31-455d-8b37-c302bf946472","Type":"ContainerStarted","Data":"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"}
Nov 03 20:53:20.703084 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:20.702701    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" event={"ID":"b9fee32f-9d31-455d-8b37-c302bf946472","Type":"ContainerStarted","Data":"bf4592ee46d15863accbf36c0ad65b62fe0cca1b26a4b512aba654c556611a5c"}
Nov 03 20:53:20.820040 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:20.819988    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podStartSLOduration=1.819975946 podStartE2EDuration="1.819975946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:53:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:53:20.818625225 +0000 UTC m=+815.269994402" watchObservedRunningTime="2026-11-03 20:53:20.819975946 +0000 UTC m=+815.271345119"
Nov 03 20:53:21.633271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:21.633228    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:24.627948 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:24.627881    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:29.450487 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:29.450407    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:29.633545 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:29.633510    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:29.651407 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:29.651380    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:29.768551 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:29.768471    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 20:53:34.338384 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:34.338343    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:37.003781 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:53:37.003733    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:53:39.126134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:39.126095    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:43.040224 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:43.040192    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:43.936314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:43.936280    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:48.847235 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:48.847197    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:53.628107 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:53.628066    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:53:58.508756 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:53:58.508725    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:02.047228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:02.047193    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:03.434535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:03.434496    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:07.008756 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:54:07.008697    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:54:07.832941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.832909    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 20:54:07.836765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.836747    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:07.889286 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.889253    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"default-dockercfg-2qc48\""
Nov 03 20:54:07.909943 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.909911    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"premium-w1-d01cef-kserve-self-signed-certs\""
Nov 03 20:54:07.934188 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.934150    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:07.938711 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.938689    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"kube-root-ca.crt\""
Nov 03 20:54:07.938855 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.938782    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"openshift-service-ca.crt\""
Nov 03 20:54:07.941421 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:07.941401    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 20:54:08.005409 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005363    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2nzx\" (UniqueName: \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.005635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005433    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.005635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005551    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.005635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005597    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.005798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005651    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.005798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.005680    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106246    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106329    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2nzx\" (UniqueName: \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106365    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106411    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106426    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106797 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106462    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106995 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106816    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106995 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106830    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.106995 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.106877    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.109320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.109295    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.109568 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.109539    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.166666 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.166633    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2nzx\" (UniqueName: \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\") pod \"premium-w1-d01cef-kserve-549c56cf8d-gtt7s\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") " pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.346208 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.346176    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:08.449106 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.449075    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:08.825910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:08.825887    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 20:54:08.827693 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:54:08.827667    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18276f96_22f2_463b_96c2_7b09d2dd9914.slice/crio-d4e862c1aba368c28d9a22ac30c971cada6c66557fa3a8f8254971007a401664 WatchSource:0}: Error finding container d4e862c1aba368c28d9a22ac30c971cada6c66557fa3a8f8254971007a401664: Status 404 returned error can't find the container with id d4e862c1aba368c28d9a22ac30c971cada6c66557fa3a8f8254971007a401664
Nov 03 20:54:09.006518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:09.006479    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" event={"ID":"18276f96-22f2-463b-96c2-7b09d2dd9914","Type":"ContainerStarted","Data":"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"}
Nov 03 20:54:09.006518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:09.006521    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" event={"ID":"18276f96-22f2-463b-96c2-7b09d2dd9914","Type":"ContainerStarted","Data":"d4e862c1aba368c28d9a22ac30c971cada6c66557fa3a8f8254971007a401664"}
Nov 03 20:54:09.126602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:09.126496    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podStartSLOduration=2.126482601 podStartE2EDuration="2.126482601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:54:09.123694193 +0000 UTC m=+863.575063370" watchObservedRunningTime="2026-11-03 20:54:09.126482601 +0000 UTC m=+863.577851778"
Nov 03 20:54:09.450169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:09.450145    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:11.322822 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:11.322788    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:13.410542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:13.410505    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:15.960297 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:15.960262    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:18.058326 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:18.058284    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:18.449433 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:18.449401    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:18.462104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:18.462073    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:18.919422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:18.919341    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:19.085268 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:19.085237    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 20:54:22.923067 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:22.923010    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:27.830609 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:27.830539    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:32.644901 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:32.644865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:34.872317 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:34.872266    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:37.003135 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:54:37.003100    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:54:37.531309 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:37.531280    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:42.273892 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:42.273857    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:45.813920 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:45.813885    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:54:45.817212 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:45.817189    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:54:45.827626 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:45.827606    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:54:45.830935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:45.830915    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:54:47.284597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:47.284527    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:49.034934 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.034900    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 20:54:49.039024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.039005    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057512 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057487    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057647 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057530    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057647 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057585    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057653    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057729 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057675    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057881 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057857    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.057935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.057919    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.058030 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.058007    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.058067 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.058052    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r98n7\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.091938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.091901    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-e22abf-openshift-default-dockercfg-8q8f9\""
Nov 03 20:54:49.144905 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.144874    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 20:54:49.159128 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159099    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159161    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159196    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159226    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r98n7\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159263    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159316    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159348    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159413    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159838 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159623    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.159838 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.159634    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.160063 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.160013    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.160147 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.160080    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.160735 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.160713    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.162875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.162849    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.163200 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.163172    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.220257 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.220227    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r98n7\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.220448 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.220430    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.352746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.352672    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:49.781727 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.781594    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 20:54:49.784581 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:54:49.784538    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7670773_7dc2_42c1_872e_410ee798d9f7.slice/crio-75bd52bc4fd3d8b082629abf5f8b987ca86bea2d4baff4133960e1eeb4ac028f WatchSource:0}: Error finding container 75bd52bc4fd3d8b082629abf5f8b987ca86bea2d4baff4133960e1eeb4ac028f: Status 404 returned error can't find the container with id 75bd52bc4fd3d8b082629abf5f8b987ca86bea2d4baff4133960e1eeb4ac028f
Nov 03 20:54:49.786886 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.786856    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:54:49.786962 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.786920    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:54:49.786962 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:49.786949    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:54:50.321586 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.320752    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" event={"ID":"c7670773-7dc2-42c1-872e-410ee798d9f7","Type":"ContainerStarted","Data":"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"}
Nov 03 20:54:50.321586 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.320833    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" event={"ID":"c7670773-7dc2-42c1-872e-410ee798d9f7","Type":"ContainerStarted","Data":"75bd52bc4fd3d8b082629abf5f8b987ca86bea2d4baff4133960e1eeb4ac028f"}
Nov 03 20:54:50.353613 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.353545    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:50.355252 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.355227    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:50.355389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.355275    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:50.443928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:50.443872    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podStartSLOduration=2.443858951 podStartE2EDuration="2.443858951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:54:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:54:50.442642248 +0000 UTC m=+904.894011426" watchObservedRunningTime="2026-11-03 20:54:50.443858951 +0000 UTC m=+904.895228127"
Nov 03 20:54:51.353227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:51.353193    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:51.353634 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:51.353243    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:52.282112 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:52.282071    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:52.353791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:52.353447    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:52.353791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:52.353505    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:53.039920 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:53.039885    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:54:53.354023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:53.353938    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:53.354023 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:53.354004    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:54.353621 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:54.353586    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:54.353804 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:54.353639    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:55.353358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:55.353264    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 03 20:54:55.353358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:55.353323    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 03 20:54:56.470969 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:56.470926    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:56.471940 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:56.471918    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:56.472948 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:56.472928    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 20:54:57.427433 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:54:57.427398    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:02.120941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:02.120908    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:07.003473 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:55:07.003428    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:55:07.158520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:07.158482    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:11.680908 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:11.680874    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:11.939693 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:11.939616    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:16.308080 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:16.308046    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:16.842830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:16.842799    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:18.473960 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.473927    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 20:55:18.477893 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.477873    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.527897 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.527873    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528037 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.527935    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528037 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528005    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528047    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528073    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528109    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528257 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528211    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528257 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528249    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.528357 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.528271    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmzp\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.529906 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.529876    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-89bb58-openshift-default-dockercfg-26hlr\""
Nov 03 20:55:18.582137 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.582112    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 20:55:18.628951 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.628919    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.628968    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.628988    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629007    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmzp\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629050    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629101    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629366 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629124    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629366 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629162    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629366 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629199    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629500 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629475    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629611 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629586    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629680 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629649    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.629737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629693    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.630013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.629832    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.631607 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.631586    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.631847 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.631830    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.689434 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.689405    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.689574 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.689523    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmzp\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:18.792220 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:18.792141    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:19.249694 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.249667    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 20:55:19.250839 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:55:19.250814    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7219df0e_e1e5_4e00_bf21_0f60ea3f865d.slice/crio-3946268fb5bd7bda58ccdc6b2269efad440eeba2976a84a2f231d96f2e47c5a3 WatchSource:0}: Error finding container 3946268fb5bd7bda58ccdc6b2269efad440eeba2976a84a2f231d96f2e47c5a3: Status 404 returned error can't find the container with id 3946268fb5bd7bda58ccdc6b2269efad440eeba2976a84a2f231d96f2e47c5a3
Nov 03 20:55:19.252935 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.252903    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:19.253020 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.252963    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:19.253020 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.252990    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:19.546725 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.546641    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" event={"ID":"7219df0e-e1e5-4e00-bf21-0f60ea3f865d","Type":"ContainerStarted","Data":"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"}
Nov 03 20:55:19.546725 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.546708    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" event={"ID":"7219df0e-e1e5-4e00-bf21-0f60ea3f865d","Type":"ContainerStarted","Data":"3946268fb5bd7bda58ccdc6b2269efad440eeba2976a84a2f231d96f2e47c5a3"}
Nov 03 20:55:19.678738 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.678678    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podStartSLOduration=1.678658322 podStartE2EDuration="1.678658322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:55:19.671776035 +0000 UTC m=+934.123145211" watchObservedRunningTime="2026-11-03 20:55:19.678658322 +0000 UTC m=+934.130027501"
Nov 03 20:55:19.792864 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:19.792825    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:20.795717 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:20.795662    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 20:55:20.796308 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:20.796262    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 20:55:21.793127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:21.793097    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 20:55:21.793304 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:21.793157    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 20:55:21.849783 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:21.849750    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:22.792924 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:22.792891    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 20:55:22.793076 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:22.792952    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 20:55:23.792610 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:23.792580    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 20:55:23.793033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:23.792655    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 20:55:24.792928 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:24.792898    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 03 20:55:24.793308 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:24.792949    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 03 20:55:26.028728 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:26.028695    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:26.029239 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:26.029091    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:26.029906 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:26.029886    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 20:55:26.750476 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:26.750439    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:31.523211 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:31.523172    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:34.172679 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:34.172645    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:55:34.173218 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:34.172846    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy" containerID="cri-o://74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5" gracePeriod=30
Nov 03 20:55:35.042385 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.042348    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 20:55:35.046929 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.046911    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.071973 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.071940    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072108 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.071986    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072167 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072147    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072219 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072203    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072278 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072258    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072341 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072303    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072421    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdp22\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072542 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072525    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.072771 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.072750    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.098923 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.098896    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d01cef-openshift-default-dockercfg-bxtfs\""
Nov 03 20:55:35.151271 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.151238    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 20:55:35.173835 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173806    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173845    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173874    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173906    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173957    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.173990    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174006    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174032    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174291 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174095    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdp22\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174750 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174305    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174750 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174389    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174750 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174548    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.174750 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174641    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.175003 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.174979    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.176427 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.176405    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.176510 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.176485    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.237244 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.237207    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.237672 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.237649    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdp22\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.362829 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.362753    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:35.793671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.793640    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 20:55:35.794889 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:55:35.794860    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940ffd46_acfd_4aff_94a6_fcfe07f4bd8b.slice/crio-53fbda96f5fd0118dc32aa0b32a61150260d88c4677b30b868a21dc68b86f61a WatchSource:0}: Error finding container 53fbda96f5fd0118dc32aa0b32a61150260d88c4677b30b868a21dc68b86f61a: Status 404 returned error can't find the container with id 53fbda96f5fd0118dc32aa0b32a61150260d88c4677b30b868a21dc68b86f61a
Nov 03 20:55:35.797256 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.797225    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:35.797332 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.797289    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:35.797332 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:35.797318    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:55:36.323834 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:36.323796    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:36.670294 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:36.670213    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" event={"ID":"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b","Type":"ContainerStarted","Data":"affd672a9047aa7159582f2bce5793afba807b2efd78fd36ac44a4cabdf871e2"}
Nov 03 20:55:36.670294 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:36.670260    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" event={"ID":"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b","Type":"ContainerStarted","Data":"53fbda96f5fd0118dc32aa0b32a61150260d88c4677b30b868a21dc68b86f61a"}
Nov 03 20:55:36.854253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:36.854202    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podStartSLOduration=2.854188405 podStartE2EDuration="2.854188405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:55:36.851853974 +0000 UTC m=+951.303223153" watchObservedRunningTime="2026-11-03 20:55:36.854188405 +0000 UTC m=+951.305557653"
Nov 03 20:55:37.029653 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:55:37.029604    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:55:37.363849 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:37.363758    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:37.365036 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:37.365011    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:37.365148 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:37.365087    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:38.363830 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:38.363800    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:38.364002 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:38.363849    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:39.363127 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.363093    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:39.363290 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.363154    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:39.496836 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.496810    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:55:39.660387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660355    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660390    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660426    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660484    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660519    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660625    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660672    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660728    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.660972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660730    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:55:39.660972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660742    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:55:39.660972 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660814    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\") pod \"35145399-3608-4f72-aec4-6c27f0ebcbba\" (UID: \"35145399-3608-4f72-aec4-6c27f0ebcbba\") "
Nov 03 20:55:39.661157 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.660972    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:55:39.661157 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661066    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:55:39.661157 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661064    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:55:39.661313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661271    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.661313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661293    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.661313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661307    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35145399-3608-4f72-aec4-6c27f0ebcbba-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.661650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661318    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.661650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.661336    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.663161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.663128    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 20:55:39.663161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.663147    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:55:39.663297 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.663227    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "kube-api-access-xgckj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:55:39.663398 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.663381    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy" pod "35145399-3608-4f72-aec4-6c27f0ebcbba" (UID: "35145399-3608-4f72-aec4-6c27f0ebcbba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:55:39.734386 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.734296    5915 generic.go:356] "Generic (PLEG): container finished" podID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerID="74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5" exitCode=0
Nov 03 20:55:39.734617 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.734575    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" event={"ID":"35145399-3608-4f72-aec4-6c27f0ebcbba","Type":"ContainerDied","Data":"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"}
Nov 03 20:55:39.734854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.734636    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg" event={"ID":"35145399-3608-4f72-aec4-6c27f0ebcbba","Type":"ContainerDied","Data":"8f6a5b482211dee357372fc64bcf291282441f9aafaae2004adcb99aed190bd2"}
Nov 03 20:55:39.734854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.734669    5915 scope.go:122] "RemoveContainer" containerID="74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"
Nov 03 20:55:39.734854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.734843    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"
Nov 03 20:55:39.746047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.746028    5915 scope.go:122] "RemoveContainer" containerID="74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"
Nov 03 20:55:39.746351 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:55:39.746331    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5\": container with ID starting with 74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5 not found: ID does not exist" containerID="74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"
Nov 03 20:55:39.746432 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.746359    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5"} err="failed to get container status \"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5\": rpc error: code = NotFound desc = could not find container \"74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5\": container with ID starting with 74637c4769cd8220bd12a1be6aafd40c19d54853e87edf29b7bf77b9c6d86be5 not found: ID does not exist"
Nov 03 20:55:39.762808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.762786    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.762899 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.762818    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.762899 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.762828    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgckj\" (UniqueName: \"kubernetes.io/projected/35145399-3608-4f72-aec4-6c27f0ebcbba-kube-api-access-xgckj\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.762899 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.762838    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35145399-3608-4f72-aec4-6c27f0ebcbba-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:55:39.924103 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.924045    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:55:39.926867 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:39.926843    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-277tg"]
Nov 03 20:55:40.363699 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:40.363656    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:40.363910 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:40.363721    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:41.295763 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:41.293606    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:41.364084 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:41.364049    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:41.364358 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:41.364328    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:41.684623 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:41.684108    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" path="/var/lib/kubelet/pods/35145399-3608-4f72-aec4-6c27f0ebcbba/volumes"
Nov 03 20:55:42.364061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:42.364023    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 03 20:55:42.364731 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:42.364682    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 03 20:55:43.368808 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:43.368775    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:43.369316 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:43.369165    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:43.370027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:43.370007    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 20:55:46.307373 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:46.307336    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:48.904275 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:48.904243    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:51.072861 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:51.072827    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:55:55.930456 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:55:55.930427    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:00.735075 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:00.735036    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:05.516913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:05.516852    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:07.016426 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:56:07.016374    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:56:10.330706 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:10.330669    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:12.019098 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:12.019067    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:15.128967 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:15.128933    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:16.654379 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:16.654344    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:19.553778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.553748    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 20:56:19.554346 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.554331    5915 cpu_manager.go:414] "containerMap: removing container" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy"
Nov 03 20:56:19.554418 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.554348    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy"
Nov 03 20:56:19.554467 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.554456    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="35145399-3608-4f72-aec4-6c27f0ebcbba" containerName="istio-proxy"
Nov 03 20:56:19.558673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.558655    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571226 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571194    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571261    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571288    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571397 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571343    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571397 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571380    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8qhm\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571464 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571435    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571463    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571498 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571484    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.571629 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.571550    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.605993 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.605958    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 20:56:19.610676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.610652    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.656854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.656823    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 20:56:19.672980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.672935    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.672980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.672973    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.672981    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673006    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673058    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673095    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8qhm\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673150    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673178    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673204    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673231    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673282    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2pq\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673323    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673354    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673387    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673393    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673465    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673514    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673544    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673594    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.673635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673624    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.674208 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.673984    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.684030 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.683918    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.684030 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.683944    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.684400 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.684123    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.684682 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.684659    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.691276 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.689323    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.735241 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.735215    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.745863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.745835    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8qhm\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-98f8h\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.795717    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.795849    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.795903    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2pq\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.795942    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.795987    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.796039    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.796083    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.796122    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.797583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.796147    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.810855 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.810085    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.811065 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.810548    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.811399 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.811380    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.811511 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.811436    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.815941 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.815902    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.823252 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.823227    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.825314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.825286    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.871425 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.871387    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.871641 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.871576    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2pq\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.875758 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.875731    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:19.937680 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.931950    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:19.942583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:19.942327    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:20.324608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.323938    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 20:56:20.335134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.333008    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.335134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.333098    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.335134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.333137    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.389958 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.389379    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 20:56:20.391598 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:56:20.391446    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbfe6a1_1a4c_4d07_ac7b_ac71aa4ddfb1.slice/crio-33ee77a82fba6873c7de3e8e428cb09dfa5117f6004754bed06d117706422869 WatchSource:0}: Error finding container 33ee77a82fba6873c7de3e8e428cb09dfa5117f6004754bed06d117706422869: Status 404 returned error can't find the container with id 33ee77a82fba6873c7de3e8e428cb09dfa5117f6004754bed06d117706422869
Nov 03 20:56:20.394140 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.394103    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.394230 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.394187    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.394270 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:20.394229    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:21.086870 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.086836    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" event={"ID":"4d7e72bd-ec36-4625-a884-555f558852c3","Type":"ContainerStarted","Data":"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"}
Nov 03 20:56:21.087747 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.087710    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" event={"ID":"4d7e72bd-ec36-4625-a884-555f558852c3","Type":"ContainerStarted","Data":"5c9a302ddf704476f863559d5ce8c80a527acd97b71223e78e077e415dad93c5"}
Nov 03 20:56:21.090466 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.089033    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" event={"ID":"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1","Type":"ContainerStarted","Data":"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"}
Nov 03 20:56:21.090466 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.089075    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" event={"ID":"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1","Type":"ContainerStarted","Data":"33ee77a82fba6873c7de3e8e428cb09dfa5117f6004754bed06d117706422869"}
Nov 03 20:56:21.556587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.500352    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podStartSLOduration=2.5003270239999997 podStartE2EDuration="2.500327024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:21.2340685 +0000 UTC m=+995.685437678" watchObservedRunningTime="2026-11-03 20:56:21.500327024 +0000 UTC m=+995.951696203"
Nov 03 20:56:21.581620 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.559175    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podStartSLOduration=2.559154777 podStartE2EDuration="2.559154777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:21.436536045 +0000 UTC m=+995.887905229" watchObservedRunningTime="2026-11-03 20:56:21.559154777 +0000 UTC m=+996.010523958"
Nov 03 20:56:21.879980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.879879    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:21.880289 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.880238    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:21.880845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.880812    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:21.944584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.942208    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:21.944584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.942337    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:21.944584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:21.942379    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:22.877584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:22.877153    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:22.877584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:22.877217    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:22.937713 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:22.937680    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:22.938018 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:22.937974    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:23.877018 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:23.876983    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:23.877171 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:23.877033    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:23.940583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:23.940505    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:23.941098 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:23.940594    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:24.877002 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:24.876969    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:24.877186 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:24.877018    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:24.935584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:24.935327    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:24.935584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:24.935382    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:25.139642 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:25.139162    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:25.877258 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:25.877159    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:25.877966 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:25.877928    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:25.949877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:25.949838    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:25.950186 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:25.950153    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:26.081591 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.080020    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:26.208108 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.208069    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"]
Nov 03 20:56:26.224964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.224933    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.284803 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.284767    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"]
Nov 03 20:56:26.293048 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.293018    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lthx\" (UniqueName: \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.293333 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.293286    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.293784 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.293755    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.295491 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.295471    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 20:56:26.296004 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.295987    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 20:56:26.466645 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.441501    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.466868 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.466836    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lthx\" (UniqueName: \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.467183 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.467158    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.467771 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.443760    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.472072 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.472051    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.554206 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.554160    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lthx\" (UniqueName: \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\") pod \"maas-api-revoke-sub-278f121b-g7gl2\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.591063 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.591037    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:26.876814 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.876781    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:26.876984 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.876844    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:26.942585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.937339    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:26.942585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:26.937399    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:27.084276 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.084250    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"]
Nov 03 20:56:27.085781 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:56:27.085755    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150bb752_c130_431c_9215_48112c82ed70.slice/crio-350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9 WatchSource:0}: Error finding container 350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9: Status 404 returned error can't find the container with id 350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9
Nov 03 20:56:27.087812 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.087799    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:56:27.301153 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.301111    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2" event={"ID":"150bb752-c130-431c-9215-48112c82ed70","Type":"ContainerStarted","Data":"350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9"}
Nov 03 20:56:27.876317 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.876278    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 03 20:56:27.876487 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.876343    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 03 20:56:27.934487 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.934452    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 03 20:56:27.934703 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:27.934512    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 03 20:56:29.565857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.565814    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:29.568068 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.568049    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:29.573807 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.573783    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 20:56:29.576418 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.576395    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:29.576780 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.576762    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:29.579027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:29.579007    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 20:56:30.259721 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:30.259686    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:34.389662 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.389631    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 20:56:34.394228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.394201    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.448635 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.448608    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-499187-openshift-default-dockercfg-jsmm9\""
Nov 03 20:56:34.472869 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.472832    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473056 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.472981    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473056 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473026    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473077    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473093    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctg82\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473127    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473169 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473166    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473361 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473203    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.473361 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.473221    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.501865 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.501832    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 20:56:34.574805 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574772    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.574987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574829    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.574987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574878    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.574987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574901    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctg82\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.574987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574938    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.574987 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.574962    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575133    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575175    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575215    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575249    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575593    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575693 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575668    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575693 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575677    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.575857 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.575834    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.577813 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.577788    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.579507 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.579450    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.636413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.636381    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.636671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.636646    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctg82\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:34.712280 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:34.712248    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:35.160227 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.160192    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 20:56:35.160922 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:56:35.160896    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8190bf_c316_4f3a_bafe_517a8b3254fb.slice/crio-174c7a37421d7dc62648732241b6136ee4750db51ad2f2ec4a99b575c0ae09ed WatchSource:0}: Error finding container 174c7a37421d7dc62648732241b6136ee4750db51ad2f2ec4a99b575c0ae09ed: Status 404 returned error can't find the container with id 174c7a37421d7dc62648732241b6136ee4750db51ad2f2ec4a99b575c0ae09ed
Nov 03 20:56:35.163950 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.163916    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:35.164043 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.163995    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:35.164098 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.164048    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:35.245656 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.245619    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:35.397678 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.397643    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" event={"ID":"9e8190bf-c316-4f3a-bafe-517a8b3254fb","Type":"ContainerStarted","Data":"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"}
Nov 03 20:56:35.398177 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.397705    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" event={"ID":"9e8190bf-c316-4f3a-bafe-517a8b3254fb","Type":"ContainerStarted","Data":"174c7a37421d7dc62648732241b6136ee4750db51ad2f2ec4a99b575c0ae09ed"}
Nov 03 20:56:35.521094 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.521035    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podStartSLOduration=1.521015099 podStartE2EDuration="1.521015099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:35.518103832 +0000 UTC m=+1009.969473007" watchObservedRunningTime="2026-11-03 20:56:35.521015099 +0000 UTC m=+1009.972384278"
Nov 03 20:56:35.713193 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:35.713118    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:36.430489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:36.430450    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2" event={"ID":"150bb752-c130-431c-9215-48112c82ed70","Type":"ContainerStarted","Data":"d34175c33d10be80534aadba86ea307c3ecf786105af5593ecc0bad36f86123e"}
Nov 03 20:56:36.545403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:36.545347    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2" podStartSLOduration=1.966377938 podStartE2EDuration="10.545328735s" totalImagesPullingTime="8.578950797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:36.544627605 +0000 UTC m=+1010.995996783" watchObservedRunningTime="2026-11-03 20:56:36.545328735 +0000 UTC m=+1010.996697914"
Nov 03 20:56:36.725584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:36.721592    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 20:56:36.725584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:36.721680    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 20:56:37.028535 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:56:37.028454    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:56:37.439027 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:37.438940    5915 generic.go:356] "Generic (PLEG): container finished" podID="150bb752-c130-431c-9215-48112c82ed70" containerID="d34175c33d10be80534aadba86ea307c3ecf786105af5593ecc0bad36f86123e" exitCode=0
Nov 03 20:56:37.439485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:37.439022    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2" event={"ID":"150bb752-c130-431c-9215-48112c82ed70","Type":"ContainerDied","Data":"d34175c33d10be80534aadba86ea307c3ecf786105af5593ecc0bad36f86123e"}
Nov 03 20:56:37.723585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:37.721955    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 20:56:37.723585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:37.722014    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 20:56:38.650568 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.650535    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:38.724013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.723976    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 20:56:38.724191 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.724043    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 20:56:38.761083 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.761059    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\" (UniqueName: \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\") pod \"150bb752-c130-431c-9215-48112c82ed70\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") "
Nov 03 20:56:38.761233 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.761140    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\") pod \"150bb752-c130-431c-9215-48112c82ed70\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") "
Nov 03 20:56:38.761297 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.761230    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\") pod \"150bb752-c130-431c-9215-48112c82ed70\" (UID: \"150bb752-c130-431c-9215-48112c82ed70\") "
Nov 03 20:56:38.761641 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.761604    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca" pod "150bb752-c130-431c-9215-48112c82ed70" (UID: "150bb752-c130-431c-9215-48112c82ed70"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:56:38.761822 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.761797    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/150bb752-c130-431c-9215-48112c82ed70-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:56:38.763484 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.763463    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx" pod "150bb752-c130-431c-9215-48112c82ed70" (UID: "150bb752-c130-431c-9215-48112c82ed70"). InnerVolumeSpecName "kube-api-access-7lthx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:56:38.763579 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.763523    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca" pod "150bb752-c130-431c-9215-48112c82ed70" (UID: "150bb752-c130-431c-9215-48112c82ed70"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:56:38.863451 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.863415    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lthx\" (UniqueName: \"kubernetes.io/projected/150bb752-c130-431c-9215-48112c82ed70-kube-api-access-7lthx\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:56:38.863451 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:38.863451    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/150bb752-c130-431c-9215-48112c82ed70-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:56:39.452103 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:39.452073    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"
Nov 03 20:56:39.452103 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:39.452084    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2" event={"ID":"150bb752-c130-431c-9215-48112c82ed70","Type":"ContainerDied","Data":"350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9"}
Nov 03 20:56:39.452313 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:39.452126    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="350395a6ee1761fad3c7b2f18498d3f93b3d84cb1b1b4fe6997c4c0c5f13bbe9"
Nov 03 20:56:39.723637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:39.720902    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 20:56:39.723637 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:39.720958    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 20:56:40.014363 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:40.014234    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:40.717015 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:40.716978    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 03 20:56:40.717329 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:40.717287    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 03 20:56:41.922964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:41.921702    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:41.922964 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:41.922351    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:41.923518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:41.923226    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 20:56:45.321580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:45.316311    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:50.149982 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:50.149943    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:54.921788 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:54.921752    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:58.157612 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.157574    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-lqdcc-sgb58"]
Nov 03 20:56:58.158176 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.158160    5915 cpu_manager.go:414] "containerMap: removing container" podUID="150bb752-c130-431c-9215-48112c82ed70" containerName="revoke-keys"
Nov 03 20:56:58.158225 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.158177    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="150bb752-c130-431c-9215-48112c82ed70" containerName="revoke-keys"
Nov 03 20:56:58.158270 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.158261    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="150bb752-c130-431c-9215-48112c82ed70" containerName="revoke-keys"
Nov 03 20:56:58.163409 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.163366    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.214288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.214257    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.214415 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.214315    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnfd2\" (UniqueName: \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.214415 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.214403    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.214501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.214435    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.265433 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.265408    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vnlw8\""
Nov 03 20:56:58.268314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.268290    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lqdcc-sgb58"]
Nov 03 20:56:58.315494 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.315462    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.315673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.315513    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.315673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.315615    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.315673 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.315670    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnfd2\" (UniqueName: \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.316082 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.316060    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.316253 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.316233    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.316373 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.316354    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.377998 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.377971    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnfd2\" (UniqueName: \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\") pod \"periodic-gathering-lqdcc-sgb58\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") " pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.475876 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.475847    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:56:58.854590 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:58.854545    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lqdcc-sgb58"]
Nov 03 20:56:58.855551 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:56:58.855523    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e7ee49_5420_46f9_b1cf_b3d4bd9b0fa3.slice/crio-71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8 WatchSource:0}: Error finding container 71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8: Status 404 returned error can't find the container with id 71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8
Nov 03 20:56:59.632489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:59.632457    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lqdcc-sgb58" event={"ID":"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3","Type":"ContainerStarted","Data":"8c7f42be6e5a0488eb3e34d46648906bf73cb468f6ebe8470670d22adcb83d21"}
Nov 03 20:56:59.632489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:59.632505    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lqdcc-sgb58" event={"ID":"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3","Type":"ContainerStarted","Data":"71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8"}
Nov 03 20:56:59.724945 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:59.724914    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:56:59.748506 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:56:59.748458    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-lqdcc-sgb58" podStartSLOduration=1.748440261 podStartE2EDuration="1.748440261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:59.747037843 +0000 UTC m=+1034.198407021" watchObservedRunningTime="2026-11-03 20:56:59.748440261 +0000 UTC m=+1034.199809439"
Nov 03 20:57:03.652083 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.652048    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 20:57:03.657140 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.657114    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.673925 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.673895    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbmg\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674074 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.673949    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674074 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674014    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674074 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674070    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674261 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674206    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674261 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674240    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674371 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674315    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674371 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674343    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.674371 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.674366    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.712021 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.711994    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ca4642-openshift-default-dockercfg-gbv98\""
Nov 03 20:57:03.767011 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.766982    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 20:57:03.775863 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.775835    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.775879    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.775927    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.775963    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776025 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.775989    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776045    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbmg\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776086    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776107    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776262 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776125    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.776484 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776282    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.777189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.776580    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.777946 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.777900    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.778777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.778752    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.779072 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.779048    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.779631 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.779606    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.780077 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.780055    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.812155 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.812122    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:03.837075 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.837055    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.837659 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.837638    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbmg\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:03.976448 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:03.976416    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:04.410539 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.410505    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 20:57:04.411137 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:57:04.411112    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f6e328_9dc0_4910_8943_a50541d7876f.slice/crio-74d51e7cd890b84f7df5852b3e91ad1ec9f192dc6988dafe34575279e565b499 WatchSource:0}: Error finding container 74d51e7cd890b84f7df5852b3e91ad1ec9f192dc6988dafe34575279e565b499: Status 404 returned error can't find the container with id 74d51e7cd890b84f7df5852b3e91ad1ec9f192dc6988dafe34575279e565b499
Nov 03 20:57:04.413472 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.413435    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:04.413603 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.413518    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:04.413603 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.413580    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:04.627120 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.627087    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:04.710963 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.710932    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" event={"ID":"77f6e328-9dc0-4910-8943-a50541d7876f","Type":"ContainerStarted","Data":"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"}
Nov 03 20:57:04.711640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.711615    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" event={"ID":"77f6e328-9dc0-4910-8943-a50541d7876f","Type":"ContainerStarted","Data":"74d51e7cd890b84f7df5852b3e91ad1ec9f192dc6988dafe34575279e565b499"}
Nov 03 20:57:04.834453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.834405    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podStartSLOduration=1.834391885 podStartE2EDuration="1.834391885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:04.831870005 +0000 UTC m=+1039.283239183" watchObservedRunningTime="2026-11-03 20:57:04.834391885 +0000 UTC m=+1039.285761061"
Nov 03 20:57:04.977235 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:04.977156    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:05.978991 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:05.978962    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 03 20:57:05.979389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:05.979012    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 03 20:57:06.978216 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:06.977536    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 03 20:57:06.978510 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:06.978458    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 03 20:57:07.077033 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:57:07.076978    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:57:07.977240 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:07.977211    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 03 20:57:07.977737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:07.977262    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 03 20:57:08.977265 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:08.977228    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 03 20:57:08.977695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:08.977291    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 03 20:57:09.648485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:09.648450    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:09.977181 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:09.977147    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 03 20:57:09.977387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:09.977219    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 03 20:57:11.244994 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:11.243784    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:11.244994 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:11.244915    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:11.246105 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:11.246077    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 20:57:12.366192 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:12.366160    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:14.358221 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:14.358180    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:17.007138 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:17.007108    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:18.050903 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:18.050872    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:19.217913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:19.217844    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:21.669399 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:21.669355    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:24.206320 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:24.206275    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:25.347028 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:25.346995    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:25.543777 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:25.543739    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:27.669144 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:27.669108    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:29.247056 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:29.247023    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:34.036252 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.036203    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:34.242569 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.242531    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 20:57:34.247015 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.246992    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.353104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.353014    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 20:57:34.400939 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.400897    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.400939 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.400938    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.400964    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.400984    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.401013    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.401030    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401210 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.401196    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4pf9\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401477 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.401334    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.401477 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.401447    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.502815 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.502779    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.502967 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.502821    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.502967 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.502942    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.502986    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503023    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503125 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503041    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503125 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503082    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4pf9\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503185    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503228 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503201    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503339 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503286    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503396 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503356    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503453 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503428    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503727    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.503791 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.503726    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.505370 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.505347    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.505643 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.505625    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.566479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.566451    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4pf9\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.566675 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.566456    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:34.860889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:34.860855    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:35.175820 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.175743    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:35.294430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.294400    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 20:57:35.295696 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:57:35.295673    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4c3db1_b042_4be3_9709_d056ca8f077e.slice/crio-1b1933d51c502c0d6473f85290a520a814cd36ec0e585dd7da3b3ba4c81c23c6 WatchSource:0}: Error finding container 1b1933d51c502c0d6473f85290a520a814cd36ec0e585dd7da3b3ba4c81c23c6: Status 404 returned error can't find the container with id 1b1933d51c502c0d6473f85290a520a814cd36ec0e585dd7da3b3ba4c81c23c6
Nov 03 20:57:35.298192 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.298156    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:35.298306 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.298237    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:35.298306 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.298280    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:35.981078 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.981041    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" event={"ID":"3c4c3db1-b042-4be3-9709-d056ca8f077e","Type":"ContainerStarted","Data":"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"}
Nov 03 20:57:35.981299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:35.981104    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" event={"ID":"3c4c3db1-b042-4be3-9709-d056ca8f077e","Type":"ContainerStarted","Data":"1b1933d51c502c0d6473f85290a520a814cd36ec0e585dd7da3b3ba4c81c23c6"}
Nov 03 20:57:36.119583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:36.103868    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podStartSLOduration=2.103850661 podStartE2EDuration="2.103850661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:36.101003709 +0000 UTC m=+1070.552372878" watchObservedRunningTime="2026-11-03 20:57:36.103850661 +0000 UTC m=+1070.555219839"
Nov 03 20:57:36.861288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:36.861257    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:36.862731 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:36.862707    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 03 20:57:36.862842 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:36.862752    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 03 20:57:37.026661 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:57:37.026630    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:57:37.861160 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:37.861127    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 03 20:57:37.861345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:37.861191    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 03 20:57:38.861812 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:38.861780    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 03 20:57:38.862321 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:38.861832    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 03 20:57:38.943969 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:38.943941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:39.861587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:39.861527    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 03 20:57:39.861779 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:39.861621    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 03 20:57:40.861508 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:40.861478    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 03 20:57:40.861980 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:40.861535    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 03 20:57:41.990854 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:41.990821    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:41.992142 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:41.992120    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:41.992997 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:41.992974    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 20:57:43.816145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:43.816100    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:48.709709 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:48.709677    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:50.042413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:50.042364    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:51.133277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.133247    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 20:57:51.138510 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.138491    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210488    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210532    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210633    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210676 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210659    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210743    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210769    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210903 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210883    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.210938 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210923    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f7cc\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.211010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.210998    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.245303 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.245279    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 20:57:51.312281 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312248    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312289    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f7cc\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312356    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312417    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312457    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312497    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312521    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312640 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312604    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.312813 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312648    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.313017 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.312987    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.313171 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.313151    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.313264 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.313240    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.313326 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.313271    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.313326 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.313275    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.315181 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.315162    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.315452 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.315432    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.373069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.373036    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f7cc\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.373069 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.373058    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.453120 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.453092    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:51.882026 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.882004    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 20:57:51.883404 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:57:51.883378    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b605f4_86d4_4166_a28d_ed1a38fb490f.slice/crio-a82393ec28c326ea6f0725642cd5c27719a7a234f169fb82a444fa2f186bcd8d WatchSource:0}: Error finding container a82393ec28c326ea6f0725642cd5c27719a7a234f169fb82a444fa2f186bcd8d: Status 404 returned error can't find the container with id a82393ec28c326ea6f0725642cd5c27719a7a234f169fb82a444fa2f186bcd8d
Nov 03 20:57:51.885764 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.885730    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:51.885877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.885806    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:51.885877 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:51.885850    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:52.153387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:52.153303    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" event={"ID":"04b605f4-86d4-4166-a28d-ed1a38fb490f","Type":"ContainerStarted","Data":"6148435255fffcc9ebc2972b6f5d6f70e1db57d0250da428ec084bcdba422ef0"}
Nov 03 20:57:52.153387 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:52.153383    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" event={"ID":"04b605f4-86d4-4166-a28d-ed1a38fb490f","Type":"ContainerStarted","Data":"a82393ec28c326ea6f0725642cd5c27719a7a234f169fb82a444fa2f186bcd8d"}
Nov 03 20:57:52.275033 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:52.274969    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podStartSLOduration=1.274954257 podStartE2EDuration="1.274954257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:52.272189773 +0000 UTC m=+1086.723558951" watchObservedRunningTime="2026-11-03 20:57:52.274954257 +0000 UTC m=+1086.726323434"
Nov 03 20:57:52.453860 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:52.453832    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:53.455663 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:53.455630    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:53.456126 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:53.455687    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:53.540479 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:53.540451    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:54.454381 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:54.454349    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:54.454589 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:54.454400    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:55.453715 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:55.453621    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:55.453715 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:55.453701    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:56.453680 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:56.453641    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:56.453890 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:56.453702    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:57.453650 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:57.453618    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:57.453847 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:57.453673    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:58.429223 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:58.429178    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:57:58.454409 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:58.454379    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 03 20:57:58.454544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:58.454447    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 03 20:57:59.458123 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:59.458085    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:59.458628 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:59.458461    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:57:59.459146 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:57:59.459121    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 20:58:03.203166 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:03.203128    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:05.164688 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.164659    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 20:58:05.169339 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.169320    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.277547 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.277513    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 20:58:05.338147 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338116    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338327 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338169    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338327 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338190    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338327 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338308    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338507 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338378    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338600 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338519    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff22b\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338614    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338671 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338656    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.338778 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.338755    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.439927 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.439851    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.439927 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.439916    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.439952    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.439997    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440134 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440038    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440290 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440140    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff22b\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440290 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440193    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440290 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440231    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440286    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440300    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440350    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440440 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440412    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.440716 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.440700    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.441189 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.441164    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.442570 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.442542    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.442775 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.442757    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.502241 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.502212    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.502359 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.502324    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff22b\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:05.783765 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:05.783731    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:06.221412 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.221380    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 20:58:06.222331 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:58:06.222306    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod413aa1c3_d204_4712_8f57_45641afa1b9b.slice/crio-74de3bc97cef226d1ee07a9be6745742c6e1d8c4493f17814d16f6c67c45bece WatchSource:0}: Error finding container 74de3bc97cef226d1ee07a9be6745742c6e1d8c4493f17814d16f6c67c45bece: Status 404 returned error can't find the container with id 74de3bc97cef226d1ee07a9be6745742c6e1d8c4493f17814d16f6c67c45bece
Nov 03 20:58:06.224845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.224814    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:06.224949 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.224877    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:06.224949 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.224906    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:06.243374 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.243347    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" event={"ID":"413aa1c3-d204-4712-8f57-45641afa1b9b","Type":"ContainerStarted","Data":"74de3bc97cef226d1ee07a9be6745742c6e1d8c4493f17814d16f6c67c45bece"}
Nov 03 20:58:06.312345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:06.312314    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:07.028483 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:58:07.028435    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:58:07.184738 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.184676    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:07.251250 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.251213    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" event={"ID":"413aa1c3-d204-4712-8f57-45641afa1b9b","Type":"ContainerStarted","Data":"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"}
Nov 03 20:58:07.427465 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.427344    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podStartSLOduration=2.427322708 podStartE2EDuration="2.427322708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:07.426110497 +0000 UTC m=+1101.877479678" watchObservedRunningTime="2026-11-03 20:58:07.427322708 +0000 UTC m=+1101.878691886"
Nov 03 20:58:07.784523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.784495    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:07.785706 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.785683    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:07.785816 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:07.785727    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:08.041608 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:08.041477    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:08.784349 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:08.784317    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:08.784787 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:08.784371    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:09.784679 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:09.784645    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:09.785062 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:09.784699    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:10.784693 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:10.784658    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:10.785170 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:10.784720    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:11.784604 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:11.784545    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:11.784780 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:11.784630    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:12.232430 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:12.232383    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal" podUID="fd3e66eb7ec8c0db50f85619df6599ce" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 20:58:12.717042 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:12.717009    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:12.784553 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:12.784518    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 03 20:58:12.784742 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:12.784591    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 03 20:58:12.910036 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:12.910000    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:13.789731 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:13.789694    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:13.790130 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:13.790101    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:13.791126 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:13.791106    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 20:58:17.363607 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:17.363574    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:17.717895 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:17.717865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:21.570314 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:21.570275    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:22.109957 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:22.109923    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:22.636598 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:22.636545    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:26.307455 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:26.307422    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:26.450056 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:26.450022    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:27.451014 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:27.450972    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:29.171616 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.171583    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 20:58:29.176745 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.176727    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.261871 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.261831    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:29.280509 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.280488    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"distinct-w2-862138-kserve-self-signed-certs\""
Nov 03 20:58:29.283619 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.283599    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 20:58:29.351509 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.351467    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.352141 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.352110    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.352405 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.352385    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.352617 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.352600    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2s6c\" (UniqueName: \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.352733 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.352718    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.367848 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.367815    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469274 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469243    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469311    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469335    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469384    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2s6c\" (UniqueName: \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469404    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469463 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469446    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469798 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469780    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469899 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469871    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.469971 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.469886    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.471902 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.471873    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.472104 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.472087    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.533694 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.533596    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2s6c\" (UniqueName: \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\") pod \"distinct-w2-862138-kserve-869bdc45bc-qrrbz\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") " pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:29.790402 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:29.790330    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:30.198954 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:30.198925    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 20:58:30.200245 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:58:30.200216    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod593ad785_80ee_4929_966e_818906ba07c5.slice/crio-e8f3ea7330dd098267624d7113a37952d2adce7bac38ed70b7977cd5c56a79dd WatchSource:0}: Error finding container e8f3ea7330dd098267624d7113a37952d2adce7bac38ed70b7977cd5c56a79dd: Status 404 returned error can't find the container with id e8f3ea7330dd098267624d7113a37952d2adce7bac38ed70b7977cd5c56a79dd
Nov 03 20:58:30.416837 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:30.416795    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" event={"ID":"593ad785-80ee-4929-966e-818906ba07c5","Type":"ContainerStarted","Data":"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"}
Nov 03 20:58:30.417007 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:30.416848    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" event={"ID":"593ad785-80ee-4929-966e-818906ba07c5","Type":"ContainerStarted","Data":"e8f3ea7330dd098267624d7113a37952d2adce7bac38ed70b7977cd5c56a79dd"}
Nov 03 20:58:30.535445 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:30.535393    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podStartSLOduration=1.53537877 podStartE2EDuration="1.53537877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:30.532355316 +0000 UTC m=+1124.983724493" watchObservedRunningTime="2026-11-03 20:58:30.53537877 +0000 UTC m=+1124.986747947"
Nov 03 20:58:30.791139 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:30.791058    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:32.224345 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:32.224311    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:35.452535 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:35.452504    5915 generic.go:356] "Generic (PLEG): container finished" podID="b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" containerID="8c7f42be6e5a0488eb3e34d46648906bf73cb468f6ebe8470670d22adcb83d21" exitCode=0
Nov 03 20:58:35.452961 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:35.452583    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lqdcc-sgb58" event={"ID":"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3","Type":"ContainerDied","Data":"8c7f42be6e5a0488eb3e34d46648906bf73cb468f6ebe8470670d22adcb83d21"}
Nov 03 20:58:36.611339 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.611310    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:58:36.662389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662356    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\") pod \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") "
Nov 03 20:58:36.662553 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662427    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\") pod \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") "
Nov 03 20:58:36.662670 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662593    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\" (UniqueName: \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\") pod \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") "
Nov 03 20:58:36.662726 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662671    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path" pod "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" (UID: "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:58:36.662795 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662722    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\") pod \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\" (UID: \"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3\") "
Nov 03 20:58:36.662795 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.662777    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle" pod "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" (UID: "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:58:36.663111 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.663073    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle" pod "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" (UID: "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:58:36.663248 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.663221    5915 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-trusted-ca-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:58:36.663248 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.663237    5915 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-archives-path\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:58:36.663248 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.663246    5915 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-service-ca-bundle\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:58:36.664845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.664821    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2" pod "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" (UID: "b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3"). InnerVolumeSpecName "kube-api-access-mnfd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:58:36.775090 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:36.775000    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnfd2\" (UniqueName: \"kubernetes.io/projected/b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3-kube-api-access-mnfd2\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:58:37.004361 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.004258    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:37.050324 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:58:37.050238    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:58:37.255779 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.255749    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 20:58:37.256573 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.256537    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" containerName="insights-gathering"
Nov 03 20:58:37.256694 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.256593    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" containerName="insights-gathering"
Nov 03 20:58:37.256737 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.256712    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3" containerName="insights-gathering"
Nov 03 20:58:37.260330 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.260314    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.352763 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.352689    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:37.362412 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.362382    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"distinct-w6-12a423-kserve-self-signed-certs\""
Nov 03 20:58:37.362688 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.362659    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"openshift-service-ca.crt\""
Nov 03 20:58:37.362688 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.362684    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"kube-root-ca.crt\""
Nov 03 20:58:37.362845 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.362666    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"default-dockercfg-5rvzg\""
Nov 03 20:58:37.365978 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.365960    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408501    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408620    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408677    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjtd\" (UniqueName: \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408810    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408844    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.414744 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.408966    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.481587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.478686    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lqdcc-sgb58" event={"ID":"b0e7ee49-5420-46f9-b1cf-b3d4bd9b0fa3","Type":"ContainerDied","Data":"71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8"}
Nov 03 20:58:37.481587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.478751    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71ccac75f0175b93ea3c3c12d679bf0686c72a6a027a8485de88ea8a07cb7ea8"
Nov 03 20:58:37.481587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.478848    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lqdcc-sgb58"
Nov 03 20:58:37.509768 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509739    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.509918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509783    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.509918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509814    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjtd\" (UniqueName: \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.509918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509852    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.509918 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509877    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.510153 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.509943    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.510281 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.510261    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.510337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.510297    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.510376 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.510350    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.512489 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.512469    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.512695 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.512678    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.570487 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.570463    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjtd\" (UniqueName: \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\") pod \"distinct-w6-12a423-kserve-57d67766fd-dmmpl\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.572389 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.572365    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:37.961735 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:37.961709    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 20:58:37.963195 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:58:37.963165    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dd74d93_03e7_4d59_b5d2_53b539d55d03.slice/crio-f5bcee19b55fc3d8556678e38674f6586ae8ad4e9baa315776415cdab5cc475e WatchSource:0}: Error finding container f5bcee19b55fc3d8556678e38674f6586ae8ad4e9baa315776415cdab5cc475e: Status 404 returned error can't find the container with id f5bcee19b55fc3d8556678e38674f6586ae8ad4e9baa315776415cdab5cc475e
Nov 03 20:58:38.434148 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.434044    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:38.512299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.512260    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" event={"ID":"8dd74d93-03e7-4d59-b5d2-53b539d55d03","Type":"ContainerStarted","Data":"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"}
Nov 03 20:58:38.513518 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.513489    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" event={"ID":"8dd74d93-03e7-4d59-b5d2-53b539d55d03","Type":"ContainerStarted","Data":"f5bcee19b55fc3d8556678e38674f6586ae8ad4e9baa315776415cdab5cc475e"}
Nov 03 20:58:38.573687 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.573634    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:38.581939 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.581907    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:38.635973 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:38.635922    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podStartSLOduration=1.635904333 podStartE2EDuration="1.635904333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:38.632843558 +0000 UTC m=+1133.084212737" watchObservedRunningTime="2026-11-03 20:58:38.635904333 +0000 UTC m=+1133.087273533"
Nov 03 20:58:39.790766 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:39.790734    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:39.806541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:39.806513    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:40.558833 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:40.558804    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 20:58:41.903889 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:41.903858    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:46.724422 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:46.724392    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:47.573403 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:47.573365    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:47.585699 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:47.585671    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:47.604244 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:47.604220    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 20:58:51.176061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.176027    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 20:58:51.180386 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.180368    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212202 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212172    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212347 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212213    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212370    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212417 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212409    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9lmp\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212502    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212576 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212568    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212630 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212588    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212667    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.212720 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.212699    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.284923 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.284880    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 20:58:51.314145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314103    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314341 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314165    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314341 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314267    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314341 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314295    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9lmp\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314351    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314413    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314440    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314523 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314480    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314759 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314526    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314759 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314669    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.314996 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.314911    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.315115 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.315100    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.315359 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.315334    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.315428 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.315343    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.317376 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.317350    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.320092 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.320066    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.375902 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.375874    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.376003 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.375923    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9lmp\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.494118 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.494080    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:51.531974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.531942    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:51.926525 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.926420    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 20:58:51.928911 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:58:51.928884    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbf175c_323a_4c4b_830b_547a3cbee8ad.slice/crio-50ee6568279f36a03eec840e8e2a88427ee0913fc09b8df04fac30449db550fa WatchSource:0}: Error finding container 50ee6568279f36a03eec840e8e2a88427ee0913fc09b8df04fac30449db550fa: Status 404 returned error can't find the container with id 50ee6568279f36a03eec840e8e2a88427ee0913fc09b8df04fac30449db550fa
Nov 03 20:58:51.931286 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.931258    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:51.931356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.931322    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:51.931356 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:51.931349    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:52.685164 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:52.685130    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" event={"ID":"fbbf175c-323a-4c4b-830b-547a3cbee8ad","Type":"ContainerStarted","Data":"d9d968154e9c6b569fec4f3ac63e7187b7466241cd5342364153da5c7df094f8"}
Nov 03 20:58:52.685544 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:52.685185    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" event={"ID":"fbbf175c-323a-4c4b-830b-547a3cbee8ad","Type":"ContainerStarted","Data":"50ee6568279f36a03eec840e8e2a88427ee0913fc09b8df04fac30449db550fa"}
Nov 03 20:58:52.920580 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:52.910529    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podStartSLOduration=1.910508705 podStartE2EDuration="1.910508705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:52.853979094 +0000 UTC m=+1147.305348289" watchObservedRunningTime="2026-11-03 20:58:52.910508705 +0000 UTC m=+1147.361877882"
Nov 03 20:58:53.494457 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:53.494423    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:53.495834 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:53.495810    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:53.495934 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:53.495861    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:54.494895 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:54.494862    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:54.495300 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:54.494914    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:55.495299 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:55.495265    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:55.495684 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:55.495314    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:56.442895 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:56.442855    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:58:56.503627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:56.501701    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:56.503627 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:56.501803    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:57.495039 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:57.495006    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:57.495233 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:57.495054    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:58.494363 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:58.494329    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 03 20:58:58.495013 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:58.494405    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 03 20:58:59.499154 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:59.499124    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:59.499643 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:59.499415    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:58:59.500260 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:58:59.500244    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 20:59:01.053284 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:01.053249    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:01.327352 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:01.327265    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:05.280401 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.280368    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 20:59:05.284591 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.284547    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.391873 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.391836    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 20:59:05.451125 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451084    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451287 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451144    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451287 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451252    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451379 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451288    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451379 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451322    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451402    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451483    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451442 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451513    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.451707 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.451632    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7sh4\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553496 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553417    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553496 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553476    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553538    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553597    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553746 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553681    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7sh4\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553764    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553808    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.553913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553881    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554097 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553921    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554097 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.553980    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554097 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.554025    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.554458    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554501 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.554496    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.554727 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.554703    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.556926 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.556893    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.557060 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.557044    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.615121 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.615089    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.615330 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.615308    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7sh4\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:05.898675 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:05.898602    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:06.235868 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.235832    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:06.344172 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.344133    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 20:59:06.373700 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:59:06.373671    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50bcc61_71b1_472c_933b_298d289cd55a.slice/crio-e421978dae3c464b83a2fc757ae3f0dfb661f9e9d2af6d9dac826734afe1d1f3 WatchSource:0}: Error finding container e421978dae3c464b83a2fc757ae3f0dfb661f9e9d2af6d9dac826734afe1d1f3: Status 404 returned error can't find the container with id e421978dae3c464b83a2fc757ae3f0dfb661f9e9d2af6d9dac826734afe1d1f3
Nov 03 20:59:06.376011 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.375978    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:06.376095 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.376054    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:06.376095 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.376081    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:06.816131 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.816095    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" event={"ID":"a50bcc61-71b1-472c-933b-298d289cd55a","Type":"ContainerStarted","Data":"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"}
Nov 03 20:59:06.816342 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.816158    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" event={"ID":"a50bcc61-71b1-472c-933b-298d289cd55a","Type":"ContainerStarted","Data":"e421978dae3c464b83a2fc757ae3f0dfb661f9e9d2af6d9dac826734afe1d1f3"}
Nov 03 20:59:06.902675 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.902637    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:06.939078 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:06.939012    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podStartSLOduration=1.938998068 podStartE2EDuration="1.938998068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:59:06.93533356 +0000 UTC m=+1161.386702730" watchObservedRunningTime="2026-11-03 20:59:06.938998068 +0000 UTC m=+1161.390367320"
Nov 03 20:59:07.030784 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:59:07.030745    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:59:07.925176 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:07.925136    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:07.925681 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:07.925203    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:08.923997 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:08.923960    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:08.924278 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:08.924249    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:09.922934 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:09.922893    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:09.923458 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:09.922954    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:10.906955 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:10.906916    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:10.907241 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:10.907211    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:11.052464 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:11.052430    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:11.923697 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:11.923663    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:11.923879 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:11.923711    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:12.899047 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:12.899014    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body=
Nov 03 20:59:12.899438 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:12.899076    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused"
Nov 03 20:59:13.057717 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:13.057683    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:13.904103 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:13.904075    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:13.904521 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:13.904505    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:13.905236 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:13.905216    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 20:59:15.824745 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:15.824709    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:17.706545 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:17.706509    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:20.621010 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:20.620926    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:22.456937 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:22.456906    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:25.414959 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:25.414880    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:30.249843 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:30.249802    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:30.780174 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:30.780140    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:35.024159 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:35.024120    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:37.049603 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.035638    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 20:59:37.078230 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.078191    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 20:59:37.112581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.100352    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.112581 ip-10-0-141-248 kubenswrapper[5915]: E1103 20:59:37.101355    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 20:59:37.154978 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.154941    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.165584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.161237    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202074    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202157    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202292    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202320    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8559n\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202416    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202486    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202527    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202546    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.212602 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.202609    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.219391 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.219355    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 20:59:37.303952 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.303844    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.303952 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.303902    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.303952 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.303933    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.304471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.304143    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.304999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.304858    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.304999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.304439    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.304999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.304474    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.304999 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.304787    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305036    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305097    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305127    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8559n\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305170    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305226    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305277 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305275    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305306    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305333    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305365    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305471 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305401    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nqmv\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.305689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305474    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305518    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.305689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.305547    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.306168 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.306129    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.306408 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.306382    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.308754 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.308689    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.308754 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.308709    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.366944 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.366903    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.367130 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.367103    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8559n\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.407508 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407467    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407704 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407549    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407704 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407695    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407813 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407752    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407813 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407807    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407853    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.407913 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407884    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407908    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408024 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.407949    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nqmv\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.408136    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408413 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.408341    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408615 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.408431    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408684 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.408615    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.408792 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.408773    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.410520 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.410494    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.411020 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.410999    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.468974 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.468939    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.469161 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.468951    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nqmv\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:37.518032 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.517998    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:37.533760 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:37.532858    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:38.062485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.061077    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.062485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.061170    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.062485 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.061214    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.065504 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.065274    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 20:59:38.079366 ip-10-0-141-248 kubenswrapper[5915]: W1103 20:59:38.079328    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583e6d5b_ebd6_43e7_a853_863475376c3d.slice/crio-b9fdfe382ebd64acc78ec7fcd3314d4eccccea12f72b63b4049e9c69b73eb195 WatchSource:0}: Error finding container b9fdfe382ebd64acc78ec7fcd3314d4eccccea12f72b63b4049e9c69b73eb195: Status 404 returned error can't find the container with id b9fdfe382ebd64acc78ec7fcd3314d4eccccea12f72b63b4049e9c69b73eb195
Nov 03 20:59:38.081832 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.081798    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.081939 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.081910    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.081986 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.081956    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.090349 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.090322    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 20:59:38.254265 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.254225    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" event={"ID":"d47c5e43-c802-4ba3-a503-b33787be0662","Type":"ContainerStarted","Data":"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"}
Nov 03 20:59:38.254530 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.254509    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" event={"ID":"d47c5e43-c802-4ba3-a503-b33787be0662","Type":"ContainerStarted","Data":"677feb7c5898316065678ffff211e4c7a444ddd819fae24bbafa8809a1c19dd4"}
Nov 03 20:59:38.257760 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.257738    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" event={"ID":"583e6d5b-ebd6-43e7-a853-863475376c3d","Type":"ContainerStarted","Data":"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"}
Nov 03 20:59:38.257959 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.257886    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" event={"ID":"583e6d5b-ebd6-43e7-a853-863475376c3d","Type":"ContainerStarted","Data":"b9fdfe382ebd64acc78ec7fcd3314d4eccccea12f72b63b4049e9c69b73eb195"}
Nov 03 20:59:38.381541 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.381434    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podStartSLOduration=2.381418697 podStartE2EDuration="2.381418697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:59:38.378942118 +0000 UTC m=+1192.830311294" watchObservedRunningTime="2026-11-03 20:59:38.381418697 +0000 UTC m=+1192.832788132"
Nov 03 20:59:38.512244 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.506817    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podStartSLOduration=2.5067974140000002 podStartE2EDuration="2.506797414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:59:38.501172188 +0000 UTC m=+1192.952541367" watchObservedRunningTime="2026-11-03 20:59:38.506797414 +0000 UTC m=+1192.958166593"
Nov 03 20:59:38.519102 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.519032    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:38.533470 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:38.533447    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:39.524581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:39.523798    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:39.524581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:39.523863    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:39.537587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:39.537450    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:39.537587 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:39.537528    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:40.129584 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:40.124985    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:40.521582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:40.519408    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:40.521582 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:40.519475    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:40.543585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:40.540935    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:40.543585 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:40.541011    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:41.521581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:41.519912    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:41.521581 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:41.519971    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:41.539597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:41.537323    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:41.539597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:41.537381    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:42.519583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:42.519016    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:42.519583 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:42.519079    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:42.541032 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:42.540992    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:42.541321 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:42.541290    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:43.522055 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:43.522016    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:43.522689 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:43.522656    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:43.541037 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:43.541004    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:43.541288 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:43.541259    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:44.519364 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:44.519327    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Nov 03 20:59:44.519594 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:44.519395    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Nov 03 20:59:44.533894 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:44.533856    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Nov 03 20:59:44.534344 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:44.533895    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Nov 03 20:59:45.126456 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.126424    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:45.744354 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.744314    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:45.746183 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.746158    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:45.747551 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.747527    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 20:59:45.761875 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.761851    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:45.763145 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.763124    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:45.764075 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.764054    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 20:59:45.867462 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.867432    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:59:45.870597 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.870573    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 20:59:45.881028 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.881006    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:59:45.884410 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:45.884390    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 20:59:46.802283 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:46.796056    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:50.082152 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:50.082122    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:54.908061 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:54.908019    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 20:59:59.765337 ip-10-0-141-248 kubenswrapper[5915]: I1103 20:59:59.765299    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:00.195254 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.195169    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:00.200432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.200411    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.206222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.206196    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:00.210734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.210711    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.219871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.219844    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:00.224126 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.224109    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.288663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.288620    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:00.289856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.289834    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:00.306492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.306467    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.306684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.306506    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w8mp\" (UniqueName: \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.306750 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.306687    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.306857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.306842    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k24hd\" (UniqueName: \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.329280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.329250    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:00:00.340407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.340374    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:00.408205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408167    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k24hd\" (UniqueName: \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.408368 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408261    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.408368 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408291    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8mp\" (UniqueName: \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.408368 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408333    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zvcn\" (UniqueName: \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.408628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408370    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.408628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.408432    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.409296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.409275    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.409342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.409275    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.468476 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.468453    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k24hd\" (UniqueName: \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\") pod \"maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.468649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.468527    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w8mp\" (UniqueName: \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\") pod \"maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.510395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.510369    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zvcn\" (UniqueName: \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.510534 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.510436    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.511042 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.511023    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.514973 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.514958    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:00.525539 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.525520    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:00.573949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.573921    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zvcn\" (UniqueName: \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\") pod \"maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.636610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.631211    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:00.836642 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.836513    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:00.951644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:00.947449    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:01.023591 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:00:01.021347    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda69f89_700b_49ce_8f6e_52157bb4cfec.slice/crio-7019535b2d6c0d3d4cd11ae5715f88ac31e50ebc8edaf97d21048a882dd95306 WatchSource:0}: Error finding container 7019535b2d6c0d3d4cd11ae5715f88ac31e50ebc8edaf97d21048a882dd95306: Status 404 returned error can't find the container with id 7019535b2d6c0d3d4cd11ae5715f88ac31e50ebc8edaf97d21048a882dd95306
Nov 03 21:00:01.041923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:01.040693    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:01.431052 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:00:01.431019    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adbc3e8_efde_4c7f_9bd5_abd5b1971da9.slice/crio-9cd5ba3e16cf40b1791672c6a69c4058028fefc01267dcabbc9afd12cafa590a WatchSource:0}: Error finding container 9cd5ba3e16cf40b1791672c6a69c4058028fefc01267dcabbc9afd12cafa590a: Status 404 returned error can't find the container with id 9cd5ba3e16cf40b1791672c6a69c4058028fefc01267dcabbc9afd12cafa590a
Nov 03 21:00:01.440234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:01.440212    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:01.496846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:01.496801    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerStarted","Data":"7019535b2d6c0d3d4cd11ae5715f88ac31e50ebc8edaf97d21048a882dd95306"}
Nov 03 21:00:01.498338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:01.498314    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" event={"ID":"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9","Type":"ContainerStarted","Data":"9cd5ba3e16cf40b1791672c6a69c4058028fefc01267dcabbc9afd12cafa590a"}
Nov 03 21:00:01.499932 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:01.499901    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerStarted","Data":"e8c3e9c234dc03179d243716231d133e08dec3b9d0c3ca4773eb05bab7d82d42"}
Nov 03 21:00:02.509581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.509527    5915 generic.go:356] "Generic (PLEG): container finished" podID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerID="0553cae3c199230a6d07a0dd90081251209ec0714761370ef58419e000954257" exitCode=60
Nov 03 21:00:02.510065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.509628    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerDied","Data":"0553cae3c199230a6d07a0dd90081251209ec0714761370ef58419e000954257"}
Nov 03 21:00:02.510065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.509992    5915 scope.go:122] "RemoveContainer" containerID="0553cae3c199230a6d07a0dd90081251209ec0714761370ef58419e000954257"
Nov 03 21:00:02.512049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.511971    5915 generic.go:356] "Generic (PLEG): container finished" podID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerID="58416d7134294cadad8f06df786dfa67af2ed3a6761dd3fffe848f2ccbadc2fc" exitCode=60
Nov 03 21:00:02.512137 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.512060    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerDied","Data":"58416d7134294cadad8f06df786dfa67af2ed3a6761dd3fffe848f2ccbadc2fc"}
Nov 03 21:00:02.512300 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.512281    5915 scope.go:122] "RemoveContainer" containerID="58416d7134294cadad8f06df786dfa67af2ed3a6761dd3fffe848f2ccbadc2fc"
Nov 03 21:00:02.515072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.515048    5915 generic.go:356] "Generic (PLEG): container finished" podID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerID="569ebef8fdcc15740b216e5f781abf752cb438202f7d058c9f2384a6f24367e4" exitCode=60
Nov 03 21:00:02.515199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.515074    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" event={"ID":"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9","Type":"ContainerDied","Data":"569ebef8fdcc15740b216e5f781abf752cb438202f7d058c9f2384a6f24367e4"}
Nov 03 21:00:02.515334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:02.515276    5915 scope.go:122] "RemoveContainer" containerID="569ebef8fdcc15740b216e5f781abf752cb438202f7d058c9f2384a6f24367e4"
Nov 03 21:00:03.526752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.525949    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerStarted","Data":"d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"}
Nov 03 21:00:03.530388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.529484    5915 generic.go:356] "Generic (PLEG): container finished" podID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerID="2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8" exitCode=60
Nov 03 21:00:03.530388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.529727    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" event={"ID":"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9","Type":"ContainerDied","Data":"2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8"}
Nov 03 21:00:03.530388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.529774    5915 scope.go:122] "RemoveContainer" containerID="569ebef8fdcc15740b216e5f781abf752cb438202f7d058c9f2384a6f24367e4"
Nov 03 21:00:03.530388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.529830    5915 scope.go:122] "RemoveContainer" containerID="2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8"
Nov 03 21:00:03.530388 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:03.530145    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq_odh-ai-gateway-infra(4adbc3e8-efde-4c7f-9bd5-abd5b1971da9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9"
Nov 03 21:00:03.532470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.532445    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerStarted","Data":"84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"}
Nov 03 21:00:03.532727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.532714    5915 scope.go:122] "RemoveContainer" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"
Nov 03 21:00:03.532965 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:03.532945    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b_odh-ai-gateway-infra(ef70ba16-cb34-4c64-a29e-54a154f5761c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c"
Nov 03 21:00:03.642648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:03.642599    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" podStartSLOduration=3.6425830169999998 podStartE2EDuration="3.642583017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:03.640199192 +0000 UTC m=+1218.091568369" watchObservedRunningTime="2026-11-03 21:00:03.642583017 +0000 UTC m=+1218.093952190"
Nov 03 21:00:04.540999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.540969    5915 generic.go:356] "Generic (PLEG): container finished" podID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105" exitCode=60
Nov 03 21:00:04.541427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.541112    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerDied","Data":"84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"}
Nov 03 21:00:04.541427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.541189    5915 scope.go:122] "RemoveContainer" containerID="0553cae3c199230a6d07a0dd90081251209ec0714761370ef58419e000954257"
Nov 03 21:00:04.541542 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.541426    5915 scope.go:122] "RemoveContainer" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"
Nov 03 21:00:04.541768 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:04.541746    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b_odh-ai-gateway-infra(ef70ba16-cb34-4c64-a29e-54a154f5761c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c"
Nov 03 21:00:04.543357 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.543333    5915 generic.go:356] "Generic (PLEG): container finished" podID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerID="d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d" exitCode=60
Nov 03 21:00:04.543468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.543409    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerDied","Data":"d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"}
Nov 03 21:00:04.543645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.543629    5915 scope.go:122] "RemoveContainer" containerID="d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"
Nov 03 21:00:04.543882 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:04.543861    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8_odh-ai-gateway-infra(bda69f89-700b-49ce-8f6e-52157bb4cfec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec"
Nov 03 21:00:04.545197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.545180    5915 scope.go:122] "RemoveContainer" containerID="2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8"
Nov 03 21:00:04.545423 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:04.545406    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq_odh-ai-gateway-infra(4adbc3e8-efde-4c7f-9bd5-abd5b1971da9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9"
Nov 03 21:00:04.558759 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.558741    5915 scope.go:122] "RemoveContainer" containerID="58416d7134294cadad8f06df786dfa67af2ed3a6761dd3fffe848f2ccbadc2fc"
Nov 03 21:00:04.646544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.646518    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"]
Nov 03 21:00:04.651878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.651857    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.676464 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.676428    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"]
Nov 03 21:00:04.703747 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.703715    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:00:04.724342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.724309    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.724500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.724461    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghmk6\" (UniqueName: \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.724588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.724519    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.804755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.804682    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:04.825318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.825291    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.825443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.825345    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghmk6\" (UniqueName: \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.825443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.825377    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.825978 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.825953    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.827883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.827857    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.910584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.906602    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghmk6\" (UniqueName: \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\") pod \"maas-api-revoke-sub-3438d597-274ln\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:04.974408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:04.974377    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:05.355806 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:05.355768    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"]
Nov 03 21:00:05.356460 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:00:05.356437    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e662501_ccd6_4517_88fe_0b078fb2b206.slice/crio-ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f WatchSource:0}: Error finding container ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f: Status 404 returned error can't find the container with id ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f
Nov 03 21:00:05.555666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:05.555632    5915 scope.go:122] "RemoveContainer" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"
Nov 03 21:00:05.556081 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:05.555938    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b_odh-ai-gateway-infra(ef70ba16-cb34-4c64-a29e-54a154f5761c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c"
Nov 03 21:00:05.557316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:05.557299    5915 scope.go:122] "RemoveContainer" containerID="d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"
Nov 03 21:00:05.557591 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:05.557548    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8_odh-ai-gateway-infra(bda69f89-700b-49ce-8f6e-52157bb4cfec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec"
Nov 03 21:00:05.558602 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:05.558575    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln" event={"ID":"6e662501-ccd6-4517-88fe-0b078fb2b206","Type":"ContainerStarted","Data":"ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f"}
Nov 03 21:00:06.568430 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:06.568388    5915 generic.go:356] "Generic (PLEG): container finished" podID="6e662501-ccd6-4517-88fe-0b078fb2b206" containerID="ae367318b4c2b36619346ab27a07211eeb3fd077ecca247d33e60db58124364d" exitCode=0
Nov 03 21:00:06.568869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:06.568473    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln" event={"ID":"6e662501-ccd6-4517-88fe-0b078fb2b206","Type":"ContainerDied","Data":"ae367318b4c2b36619346ab27a07211eeb3fd077ecca247d33e60db58124364d"}
Nov 03 21:00:07.004041 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:07.004004    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:00:07.742136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.742113    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:07.864411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.864337    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\" (UniqueName: \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\") pod \"6e662501-ccd6-4517-88fe-0b078fb2b206\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") "
Nov 03 21:00:07.864411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.864372    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\") pod \"6e662501-ccd6-4517-88fe-0b078fb2b206\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") "
Nov 03 21:00:07.864664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.864628    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\") pod \"6e662501-ccd6-4517-88fe-0b078fb2b206\" (UID: \"6e662501-ccd6-4517-88fe-0b078fb2b206\") "
Nov 03 21:00:07.864965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.864942    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca" pod "6e662501-ccd6-4517-88fe-0b078fb2b206" (UID: "6e662501-ccd6-4517-88fe-0b078fb2b206"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:00:07.865190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.865134    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:07.866647 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.866618    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca" pod "6e662501-ccd6-4517-88fe-0b078fb2b206" (UID: "6e662501-ccd6-4517-88fe-0b078fb2b206"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:00:07.866752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.866682    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6" pod "6e662501-ccd6-4517-88fe-0b078fb2b206" (UID: "6e662501-ccd6-4517-88fe-0b078fb2b206"). InnerVolumeSpecName "kube-api-access-ghmk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:00:07.966805 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.966774    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghmk6\" (UniqueName: \"kubernetes.io/projected/6e662501-ccd6-4517-88fe-0b078fb2b206-kube-api-access-ghmk6\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:07.966805 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:07.966809    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e662501-ccd6-4517-88fe-0b078fb2b206-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:08.245494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:08.245458    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:08.664114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:08.664004    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln" event={"ID":"6e662501-ccd6-4517-88fe-0b078fb2b206","Type":"ContainerDied","Data":"ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f"}
Nov 03 21:00:08.664356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:08.664337    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebbb28d2caffb3bfdf6ee2f340afbf927db708cb175b5aff9c97f0bf7a85fd3f"
Nov 03 21:00:08.664469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:08.664142    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"
Nov 03 21:00:09.721646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:09.721615    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:13.407706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:13.407671    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:14.514161 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:14.514131    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:14.674366 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:14.674329    5915 scope.go:122] "RemoveContainer" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"
Nov 03 21:00:15.680446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.680422    5915 scope.go:122] "RemoveContainer" containerID="d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"
Nov 03 21:00:15.680851 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.680599    5915 scope.go:122] "RemoveContainer" containerID="2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8"
Nov 03 21:00:15.719788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.719758    5915 generic.go:356] "Generic (PLEG): container finished" podID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerID="7f5ca74ec3ef599ea67c4fe3bd25b83a8b5f2cd9d36a7433c3f1daab9e127a71" exitCode=60
Nov 03 21:00:15.719903 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.719807    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerDied","Data":"7f5ca74ec3ef599ea67c4fe3bd25b83a8b5f2cd9d36a7433c3f1daab9e127a71"}
Nov 03 21:00:15.719903 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.719853    5915 scope.go:122] "RemoveContainer" containerID="84c46c69cf52d239e6811f83713f645d1957b511477ccb77a715d9f83c43f105"
Nov 03 21:00:15.720287 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:15.720268    5915 scope.go:122] "RemoveContainer" containerID="7f5ca74ec3ef599ea67c4fe3bd25b83a8b5f2cd9d36a7433c3f1daab9e127a71"
Nov 03 21:00:15.720675 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:15.720646    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b_odh-ai-gateway-infra(ef70ba16-cb34-4c64-a29e-54a154f5761c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c"
Nov 03 21:00:16.729989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.729960    5915 generic.go:356] "Generic (PLEG): container finished" podID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerID="2452eb2c35d430395bd7abbb5fb6c945cae7c0faadef1f84cedfa54f763570c4" exitCode=60
Nov 03 21:00:16.730322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.730026    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerDied","Data":"2452eb2c35d430395bd7abbb5fb6c945cae7c0faadef1f84cedfa54f763570c4"}
Nov 03 21:00:16.730322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.730112    5915 scope.go:122] "RemoveContainer" containerID="d67f84d255835871d4631cd86b310c3c305d7947606dcd5495cb854dd837798d"
Nov 03 21:00:16.730505 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.730489    5915 scope.go:122] "RemoveContainer" containerID="2452eb2c35d430395bd7abbb5fb6c945cae7c0faadef1f84cedfa54f763570c4"
Nov 03 21:00:16.730873 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:16.730854    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8_odh-ai-gateway-infra(bda69f89-700b-49ce-8f6e-52157bb4cfec)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec"
Nov 03 21:00:16.731996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.731978    5915 generic.go:356] "Generic (PLEG): container finished" podID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerID="38fed96578712301999d39a1624172a49679f24ff7f086ba85120e388c6ad73e" exitCode=60
Nov 03 21:00:16.732066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.732018    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" event={"ID":"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9","Type":"ContainerDied","Data":"38fed96578712301999d39a1624172a49679f24ff7f086ba85120e388c6ad73e"}
Nov 03 21:00:16.732320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.732308    5915 scope.go:122] "RemoveContainer" containerID="38fed96578712301999d39a1624172a49679f24ff7f086ba85120e388c6ad73e"
Nov 03 21:00:16.732586 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:16.732540    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq_odh-ai-gateway-infra(4adbc3e8-efde-4c7f-9bd5-abd5b1971da9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9"
Nov 03 21:00:16.832150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.832131    5915 scope.go:122] "RemoveContainer" containerID="2662fc8ac1b8d367a107e264167c2a61f4cdc4329468655aac9d0208e1d13ca8"
Nov 03 21:00:16.847118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:16.847084    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:17.017816 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.017791    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:17.093623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.093592    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\" (UniqueName: \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\") pod \"ef70ba16-cb34-4c64-a29e-54a154f5761c\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") "
Nov 03 21:00:17.093798 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.093784    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\") pod \"ef70ba16-cb34-4c64-a29e-54a154f5761c\" (UID: \"ef70ba16-cb34-4c64-a29e-54a154f5761c\") "
Nov 03 21:00:17.094150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.094119    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca" pod "ef70ba16-cb34-4c64-a29e-54a154f5761c" (UID: "ef70ba16-cb34-4c64-a29e-54a154f5761c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:00:17.094294 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.094206    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef70ba16-cb34-4c64-a29e-54a154f5761c-maas-api-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:17.095858 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.095836    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd" pod "ef70ba16-cb34-4c64-a29e-54a154f5761c" (UID: "ef70ba16-cb34-4c64-a29e-54a154f5761c"). InnerVolumeSpecName "kube-api-access-k24hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:00:17.195819 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.195759    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k24hd\" (UniqueName: \"kubernetes.io/projected/ef70ba16-cb34-4c64-a29e-54a154f5761c-kube-api-access-k24hd\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:17.742094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.742052    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"
Nov 03 21:00:17.742533 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.742094    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b" event={"ID":"ef70ba16-cb34-4c64-a29e-54a154f5761c","Type":"ContainerDied","Data":"e8c3e9c234dc03179d243716231d133e08dec3b9d0c3ca4773eb05bab7d82d42"}
Nov 03 21:00:17.742533 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.742157    5915 scope.go:122] "RemoveContainer" containerID="7f5ca74ec3ef599ea67c4fe3bd25b83a8b5f2cd9d36a7433c3f1daab9e127a71"
Nov 03 21:00:17.859481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.859451    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:17.913488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.913446    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:17.916141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.916113    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-862138-29895660-8ml6b"]
Nov 03 21:00:17.973545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:17.973508    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:18.046504 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.046458    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:18.048965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.048634    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:18.157424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.157397    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:18.208012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.207983    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\" (UniqueName: \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\") pod \"bda69f89-700b-49ce-8f6e-52157bb4cfec\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") "
Nov 03 21:00:18.208165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.208088    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\") pod \"bda69f89-700b-49ce-8f6e-52157bb4cfec\" (UID: \"bda69f89-700b-49ce-8f6e-52157bb4cfec\") "
Nov 03 21:00:18.208477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.208452    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca" pod "bda69f89-700b-49ce-8f6e-52157bb4cfec" (UID: "bda69f89-700b-49ce-8f6e-52157bb4cfec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:00:18.210348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.210329    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp" pod "bda69f89-700b-49ce-8f6e-52157bb4cfec" (UID: "bda69f89-700b-49ce-8f6e-52157bb4cfec"). InnerVolumeSpecName "kube-api-access-8w8mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:00:18.309854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.309771    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\") pod \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") "
Nov 03 21:00:18.309996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.309899    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\" (UniqueName: \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\") pod \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\" (UID: \"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9\") "
Nov 03 21:00:18.310201 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.310175    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca" pod "4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" (UID: "4adbc3e8-efde-4c7f-9bd5-abd5b1971da9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:00:18.310477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.310458    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-maas-api-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:18.310522 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.310480    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w8mp\" (UniqueName: \"kubernetes.io/projected/bda69f89-700b-49ce-8f6e-52157bb4cfec-kube-api-access-8w8mp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:18.310522 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.310494    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bda69f89-700b-49ce-8f6e-52157bb4cfec-maas-api-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:18.312093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.312075    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn" pod "4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" (UID: "4adbc3e8-efde-4c7f-9bd5-abd5b1971da9"). InnerVolumeSpecName "kube-api-access-5zvcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:00:18.411836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.411800    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zvcn\" (UniqueName: \"kubernetes.io/projected/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9-kube-api-access-5zvcn\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:00:18.756150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.756124    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"
Nov 03 21:00:18.756150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.756141    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8" event={"ID":"bda69f89-700b-49ce-8f6e-52157bb4cfec","Type":"ContainerDied","Data":"7019535b2d6c0d3d4cd11ae5715f88ac31e50ebc8edaf97d21048a882dd95306"}
Nov 03 21:00:18.756715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.756197    5915 scope.go:122] "RemoveContainer" containerID="2452eb2c35d430395bd7abbb5fb6c945cae7c0faadef1f84cedfa54f763570c4"
Nov 03 21:00:18.757741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.757722    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"
Nov 03 21:00:18.757855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.757768    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq" event={"ID":"4adbc3e8-efde-4c7f-9bd5-abd5b1971da9","Type":"ContainerDied","Data":"9cd5ba3e16cf40b1791672c6a69c4058028fefc01267dcabbc9afd12cafa590a"}
Nov 03 21:00:18.779397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.779379    5915 scope.go:122] "RemoveContainer" containerID="38fed96578712301999d39a1624172a49679f24ff7f086ba85120e388c6ad73e"
Nov 03 21:00:18.923583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.923526    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:18.928584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:18.928508    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d01cef-29895660-7k6lq"]
Nov 03 21:00:19.038537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.038505    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:19.042002 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.041973    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e22abf-29895660-fmgz8"]
Nov 03 21:00:19.423510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.423399    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:19.682736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.682658    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" path="/var/lib/kubelet/pods/4adbc3e8-efde-4c7f-9bd5-abd5b1971da9/volumes"
Nov 03 21:00:19.683106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.683091    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" path="/var/lib/kubelet/pods/bda69f89-700b-49ce-8f6e-52157bb4cfec/volumes"
Nov 03 21:00:19.683462 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:19.683449    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" path="/var/lib/kubelet/pods/ef70ba16-cb34-4c64-a29e-54a154f5761c/volumes"
Nov 03 21:00:21.681889 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.681862    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:00:21.682422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682407    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682423    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682436    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682441    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682450    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682455    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682460    5915 cpu_manager.go:414] "containerMap: removing container" podUID="6e662501-ccd6-4517-88fe-0b078fb2b206" containerName="revoke-keys"
Nov 03 21:00:21.682470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682464    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e662501-ccd6-4517-88fe-0b078fb2b206" containerName="revoke-keys"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682475    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682479    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682491    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682496    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682515    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682519    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682525    5915 cpu_manager.go:414] "containerMap: removing container" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682529    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682542    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682546    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682657    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682665    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682671    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682676    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e662501-ccd6-4517-88fe-0b078fb2b206" containerName="revoke-keys"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682683    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682689    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:21.682737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.682697    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:21.687716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.687695    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.758175 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.758142    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:00:21.859843 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.859809    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.859869    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.859989    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860044    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860090    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860189 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860173    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860288 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860272    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860343    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvsrw\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.860419 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.860408    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962363    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962443    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962486    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962510    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962542    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962648    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.962743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962701    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvsrw\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.963065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962804    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.963065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.962896    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.999615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.973680    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.999615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.974334    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.999615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.981738    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.999615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.982029    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:21.999615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:21.986167    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.048888 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.048829    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.071651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.060224    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvsrw\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.089588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.075102    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.089588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.075623    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.306372 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.306219    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:22.867582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.866930    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:22.998956 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:22.989420    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:00:23.069997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:23.050439    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:23.069997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:23.050530    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:23.069997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:23.050599    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:23.846586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:23.846276    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" event={"ID":"649ea92f-4fd0-4e09-8299-8977c6383d1a","Type":"ContainerStarted","Data":"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"}
Nov 03 21:00:23.846586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:23.846386    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" event={"ID":"649ea92f-4fd0-4e09-8299-8977c6383d1a","Type":"ContainerStarted","Data":"21b65ca07ae9a3473d0e5c2a861d89970e28cded7aabaa3d210c608944d49fbd"}
Nov 03 21:00:24.080587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:24.080096    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podStartSLOduration=3.080074738 podStartE2EDuration="3.080074738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:24.035423273 +0000 UTC m=+1238.486792451" watchObservedRunningTime="2026-11-03 21:00:24.080074738 +0000 UTC m=+1238.531443916"
Nov 03 21:00:24.309180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:24.307981    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:24.310038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:24.310009    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:24.310439 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:24.310411    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:24.943488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:24.943440    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:25.308598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:25.307079    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:25.308598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:25.307159    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:26.320586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:26.306941    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:26.320586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:26.307003    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:27.312958 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:27.312918    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:27.313248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:27.313218    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:28.324015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:28.323975    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:28.324526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:28.324053    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:29.318025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:29.317986    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:29.318292 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:29.318265    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:29.943619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:29.943582    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:30.307275 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:30.307240    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 03 21:00:30.307433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:30.307294    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 03 21:00:30.964853 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:30.964756    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:31.556019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:31.555982    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:31.556594 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:31.556505    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:31.557755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:31.557731    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:00:31.742590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:31.742530    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:31.897837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:31.897760    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:34.791596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:34.791529    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:35.286257 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286222    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:00:35.286803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286789    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:35.286864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286804    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:35.286946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286938    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbc3e8-efde-4c7f-9bd5-abd5b1971da9" containerName="cleanup"
Nov 03 21:00:35.286981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286955    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda69f89-700b-49ce-8f6e-52157bb4cfec" containerName="cleanup"
Nov 03 21:00:35.286981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.286962    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef70ba16-cb34-4c64-a29e-54a154f5761c" containerName="cleanup"
Nov 03 21:00:35.290507 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.290490    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364464    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364496    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364512    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwzvj\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364543    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364682    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364739    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364787    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364807    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.364951 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.364903    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.390800 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.390769    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:00:35.465914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.465880    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.465981    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.465999    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466014    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwzvj\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466042    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466314 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466117    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466314 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466237    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466314 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466292    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466334    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466299    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466397    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466492    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466760 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466714    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.466912 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.466894    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.468599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.468579    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.468700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.468604    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.526980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.526951    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwzvj\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.526980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.526964    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:35.605320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:35.605243    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:36.045472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.045444    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:00:36.046724 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:00:36.046696    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53adb6c7_6789_4b49_9259_4af0abe2ad9c.slice/crio-b5d91f9d97db4b4ed3e4bfe5bdfbdca162ae05c08c9c61cd6da64047d95413dc WatchSource:0}: Error finding container b5d91f9d97db4b4ed3e4bfe5bdfbdca162ae05c08c9c61cd6da64047d95413dc: Status 404 returned error can't find the container with id b5d91f9d97db4b4ed3e4bfe5bdfbdca162ae05c08c9c61cd6da64047d95413dc
Nov 03 21:00:36.049289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.049259    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.049369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.049323    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.049369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.049359    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.052303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.052284    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:00:36.058053 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.058035    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.150578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.150521    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:00:36.174389 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174356    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174412    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnz8t\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174540    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174676    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174711    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174865    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174901    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.174970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.174957    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.175103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.175089    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278270    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278319    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278402    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278693 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278463    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278693 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278550    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278800 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278721    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278800 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278788    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278821    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278841    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnz8t\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.278909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278858    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.279072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.278912    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.279391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.279365    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.280181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.280155    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.280463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.279674    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.281650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.281625    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.281844 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.281825    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.339966 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.339897    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnz8t\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.345172 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.345145    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.374426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.374399    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:36.390638 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.390612    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:00:36.396301 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.396279    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.457971 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.456754    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.481922    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.481991    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482038    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482135    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482185    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczdv\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482240    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482275    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482339    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.482584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.482363    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584267    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584355    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584372    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584543    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584610    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584648    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584817 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584708    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584817 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584744    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczdv\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584817 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584769    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.584972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.584814    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.585348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.585324    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.585630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.585606    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.585732 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.585683    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.589801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.588499    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.590343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.590285    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.590909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.590885    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.646232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.646205    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.646402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.646362    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczdv\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.712745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.712717    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:36.777626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.776655    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:00:36.777626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.777233    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.777626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.777306    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.777626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:36.777343    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.032837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.032796    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" event={"ID":"01376f4d-aefb-44fc-bd45-3af8a8c295e0","Type":"ContainerStarted","Data":"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"}
Nov 03 21:00:37.033013 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.032867    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" event={"ID":"01376f4d-aefb-44fc-bd45-3af8a8c295e0","Type":"ContainerStarted","Data":"e4056811ab91188089a256f6ff6639cb411528dc4df35ad7392a5c5162c2be8e"}
Nov 03 21:00:37.036687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.036659    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" event={"ID":"53adb6c7-6789-4b49-9259-4af0abe2ad9c","Type":"ContainerStarted","Data":"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"}
Nov 03 21:00:37.036832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.036706    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" event={"ID":"53adb6c7-6789-4b49-9259-4af0abe2ad9c","Type":"ContainerStarted","Data":"b5d91f9d97db4b4ed3e4bfe5bdfbdca162ae05c08c9c61cd6da64047d95413dc"}
Nov 03 21:00:37.038473 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:00:37.038448    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:00:37.137331 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.137279    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:00:37.139391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.139358    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.139484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.139427    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.139484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.139464    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.155261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.155217    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podStartSLOduration=2.155203956 podStartE2EDuration="2.155203956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:37.152925365 +0000 UTC m=+1251.604294543" watchObservedRunningTime="2026-11-03 21:00:37.155203956 +0000 UTC m=+1251.606573191"
Nov 03 21:00:37.278368 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.278318    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podStartSLOduration=2.278302444 podStartE2EDuration="2.278302444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:37.276926486 +0000 UTC m=+1251.728295662" watchObservedRunningTime="2026-11-03 21:00:37.278302444 +0000 UTC m=+1251.729671623"
Nov 03 21:00:37.374856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.374776    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:37.376384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.376109    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:37.376384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.376165    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:37.608706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.608671    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:37.609000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.608963    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:37.609391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:37.608736    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:38.073617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.067463    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" event={"ID":"fbf276c4-45a5-4f4d-9742-73576ce2e479","Type":"ContainerStarted","Data":"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"}
Nov 03 21:00:38.073931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.073902    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" event={"ID":"fbf276c4-45a5-4f4d-9742-73576ce2e479","Type":"ContainerStarted","Data":"f8990c4f98c63ccd67301c69ac932957d468db801692eeabf8bc035c0aa397bf"}
Nov 03 21:00:38.191310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.191257    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podStartSLOduration=2.1912420519999998 podStartE2EDuration="2.191242052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:38.18747232 +0000 UTC m=+1252.638841497" watchObservedRunningTime="2026-11-03 21:00:38.191242052 +0000 UTC m=+1252.642611232"
Nov 03 21:00:38.378863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.378769    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.378863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.378839    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:38.608030 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.607993    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.608308 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.608282    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:38.719583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.718923    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:38.725584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.723915    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.725584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:38.723973    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:39.375585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.374755    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.375585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.374814    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:39.614029 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.613994    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.614303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.614270    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:39.719977 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.719940    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.720227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:39.720200    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:40.156644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.156145    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:40.383326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.383294    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.383947 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.383915    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:40.617224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.609707    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.617224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.609786    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:40.750584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.727653    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.750584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:40.727711    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:41.390348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.390301    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.391047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.391003    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:41.624645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.624595    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.624921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.624890    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:41.776394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.776223    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.776394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:41.776295    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:42.382373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.382255    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.382548 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.382372    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:42.625402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.625368    5915 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.629706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.629661    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 03 21:00:42.715589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.714916    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.715589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:42.714974    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:43.375042 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:43.375005    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.375220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:43.375071    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 03 21:00:43.713080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:43.713045    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.713621 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:43.713113    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 03 21:00:44.045347 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.045262    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:44.046183 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.046115    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:44.048493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.048467    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:00:44.647592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.645355    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:44.647592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.647117    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:44.656250 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:44.656039    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:00:45.066167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:45.066132    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:45.067102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:45.067080    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:45.067783 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:45.067763    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:00:45.410647 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:45.410568    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:50.235777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:50.235741    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:55.037853 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:55.037814    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:00:58.577879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:58.576727    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.577879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:58.577808    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.603770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:58.601856    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.771607 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:58.761641    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:59.076584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.069782    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:00:59.472606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.463263    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:59.485604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.475947    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:00:59.485604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.476026    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:59.623161 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.623103    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:01:00.075386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:00:59.639672    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Nov 03 21:01:00.075386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.048275    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:01:00.075386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.048346    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:00.075386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.048648    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:01:00.075386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.048924    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:01:00.430180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.430072    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:00.693640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.689772    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:00.732098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.732059    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:01:00.732374 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.732342    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:00.773867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.772337    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:00.852094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.794719    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:01:00.852094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:00.794787    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:01.023210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.022837    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:01.023210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.023010    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:01:01.069629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.046052    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:01.206584 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:01.205993    5915 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.157s"
Nov 03 21:01:01.206584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.206149    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:01:01.206584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.206193    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:01:01.329348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.329254    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:01:01.365773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.365744    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:01:01.404456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.404420    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:01:01.405352 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.405331    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:01:01.601481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:01.601390    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:06.367233 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:06.367199    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:07.005718 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:07.005683    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:01:11.159016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:11.158977    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:13.757601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:13.757522    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:15.915852 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:15.915820    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:20.117403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.117370    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:01:20.121863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.121840    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.203839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.203791    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:20.223886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.223854    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"distinct-2-w6-12a423-kserve-self-signed-certs\""
Nov 03 21:01:20.230366 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.230341    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339596    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339709    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339812    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339851    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339917    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtssm\" (UniqueName: \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.344228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.339946    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441527 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441450    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441527 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441498    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441618    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441652    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441715    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtssm\" (UniqueName: \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441749    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.441984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.441916    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.442116 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.442068    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.442174 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.442139    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.444146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.444129    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.444491 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.444474    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.544590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.544268    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtssm\" (UniqueName: \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\") pod \"distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") " pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.734286 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.734256    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:20.897104 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:20.897070    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:21.124151 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:01:21.124115    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb85e0994_00dc_40e4_9f7c_b54d13a34355.slice/crio-16423edf4f18ce9dfdabb1b2dbc27d138109395d1f7b71402881693b59d0ee3f WatchSource:0}: Error finding container 16423edf4f18ce9dfdabb1b2dbc27d138109395d1f7b71402881693b59d0ee3f: Status 404 returned error can't find the container with id 16423edf4f18ce9dfdabb1b2dbc27d138109395d1f7b71402881693b59d0ee3f
Nov 03 21:01:21.124493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:21.124379    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:01:21.419468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:21.419394    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" event={"ID":"b85e0994-00dc-40e4-9f7c-b54d13a34355","Type":"ContainerStarted","Data":"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"}
Nov 03 21:01:21.419468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:21.419431    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" event={"ID":"b85e0994-00dc-40e4-9f7c-b54d13a34355","Type":"ContainerStarted","Data":"16423edf4f18ce9dfdabb1b2dbc27d138109395d1f7b71402881693b59d0ee3f"}
Nov 03 21:01:21.537586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:21.537506    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podStartSLOduration=1.537492751 podStartE2EDuration="1.537492751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:21.534878235 +0000 UTC m=+1295.986247413" watchObservedRunningTime="2026-11-03 21:01:21.537492751 +0000 UTC m=+1295.988861928"
Nov 03 21:01:21.741583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:21.735105    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:22.177016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.176935    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:01:22.181304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.181286    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216590    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216750 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216639    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbprz\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216750 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216712    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216812    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216856    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216932 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216919    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.216992 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.216980    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.217039 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.217028    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.217113 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.217099    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.318767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.318731    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.318954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.318813    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.318954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.318845    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.318954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.318917    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.318973    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319004    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319138    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319186    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbprz\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319226    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.319765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319737    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.320028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.320001    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.320227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319869    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.320373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.319756    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.320601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.320579    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.323170 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.323143    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.323341 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.323318    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.363384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.363352    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:01:22.369781 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.369757    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.386277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.386253    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:01:22.391749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.391697    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbprz\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.392116 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.392092    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.420629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.420599    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.421163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.421063    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.421380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.421359    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.424706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.424685    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.424821 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.424738    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.424821 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.424789    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.424935 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.424841    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tfvd\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.425296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.425100    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.425296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.425174    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.430471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.430406    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:01:22.496775 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.496750    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:22.526742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526707    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.526893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526796    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.526893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526836    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.526893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526880    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527071 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526931    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527071 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.526987    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tfvd\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.527107    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.527209    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.527273    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.527523    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.527748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.527722    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.528076 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.528053    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.528216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.528117    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.528545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.528517    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.530120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.530095    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.531292 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.531257    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.590051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.590013    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.590218 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.590137    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tfvd\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.686787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.686713    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:22.940222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.940185    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:01:22.940461 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:01:22.940443    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e678505_5cc0_4f23_9e3c_2dfbcca7a334.slice/crio-0896f2d0ed2d04a3a986cb44868c7e44eeacf4f89eca8572c5cbc2d747eb3b2a WatchSource:0}: Error finding container 0896f2d0ed2d04a3a986cb44868c7e44eeacf4f89eca8572c5cbc2d747eb3b2a: Status 404 returned error can't find the container with id 0896f2d0ed2d04a3a986cb44868c7e44eeacf4f89eca8572c5cbc2d747eb3b2a
Nov 03 21:01:22.943325 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.943295    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.943396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.943372    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.943433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:22.943413    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:23.071708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.071682    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:01:23.073979 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:01:23.073954    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8e8d79_3820_4df6_b86b_ca93da2d5664.slice/crio-4b70461791c97733f6b6dbf2d55616ccdfb2434c939dd9590399d906c1eeeaef WatchSource:0}: Error finding container 4b70461791c97733f6b6dbf2d55616ccdfb2434c939dd9590399d906c1eeeaef: Status 404 returned error can't find the container with id 4b70461791c97733f6b6dbf2d55616ccdfb2434c939dd9590399d906c1eeeaef
Nov 03 21:01:23.077270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.077235    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:23.077330 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.077316    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:23.077371 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.077356    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:23.450885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.450850    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" event={"ID":"cd8e8d79-3820-4df6-b86b-ca93da2d5664","Type":"ContainerStarted","Data":"cffb6d455c782b5944c55b924201930c5784d1c2f73207eeabb8f61fe619717c"}
Nov 03 21:01:23.451380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.450916    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" event={"ID":"cd8e8d79-3820-4df6-b86b-ca93da2d5664","Type":"ContainerStarted","Data":"4b70461791c97733f6b6dbf2d55616ccdfb2434c939dd9590399d906c1eeeaef"}
Nov 03 21:01:23.453229 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.453202    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" event={"ID":"4e678505-5cc0-4f23-9e3c-2dfbcca7a334","Type":"ContainerStarted","Data":"bb1d9b7488d64d4508a0d57c3d28bce272cacb8ead860ea4c5942186f83760ba"}
Nov 03 21:01:23.453391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.453254    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" event={"ID":"4e678505-5cc0-4f23-9e3c-2dfbcca7a334","Type":"ContainerStarted","Data":"0896f2d0ed2d04a3a986cb44868c7e44eeacf4f89eca8572c5cbc2d747eb3b2a"}
Nov 03 21:01:23.497859 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.497816    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:23.571850 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.571789    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podStartSLOduration=1.571770104 podStartE2EDuration="1.571770104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:23.569609607 +0000 UTC m=+1298.020978785" watchObservedRunningTime="2026-11-03 21:01:23.571770104 +0000 UTC m=+1298.023139283"
Nov 03 21:01:23.687034 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.687006    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:23.688911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:23.688858    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podStartSLOduration=1.688823572 podStartE2EDuration="1.688823572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:23.688650933 +0000 UTC m=+1298.140020109" watchObservedRunningTime="2026-11-03 21:01:23.688823572 +0000 UTC m=+1298.140214362"
Nov 03 21:01:24.516585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:24.513778    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:24.516585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:24.513847    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:24.688021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:24.687991    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:24.688182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:24.688039    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:25.497275 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:25.497239    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:25.497448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:25.497296    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:25.687370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:25.687339    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:25.687761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:25.687399    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:25.843492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:25.834487    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:26.498066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:26.498035    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:26.498255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:26.498090    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:26.687176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:26.687144    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:26.687352 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:26.687200    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:27.497516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:27.497485    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:27.498054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:27.497546    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:27.687522 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:27.687491    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:27.687707 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:27.687541    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:28.447583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:28.447518    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:28.557601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:28.555837    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:28.557601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:28.555911    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:28.687389 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:28.687360    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:28.687644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:28.687623    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:29.497902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:29.497866    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 03 21:01:29.498088 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:29.497943    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 03 21:01:29.687579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:29.687532    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Nov 03 21:01:29.688046 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:29.687615    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Nov 03 21:01:30.735388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:30.735354    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:30.750229 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:30.750200    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:30.835238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:30.834843    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:30.835238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:30.835233    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:30.837012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:30.836989    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:01:31.013170 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:31.013078    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:31.013637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:31.013614    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:31.016969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:31.016905    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:01:31.061892 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:31.061866    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:31.836366 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:31.836337    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:33.442841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:33.442784    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.480161 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:33.480109    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.514607 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:33.504624    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.900146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:33.900102    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.295749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.295702    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.446710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.446663    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.475585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.449607    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:01:34.497273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.497214    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.532411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.505842    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:01:34.532411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.506732    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.532411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.507088    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:01:34.609693 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.584518    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:34.980584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:34.979667    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Nov 03 21:01:35.162765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.162728    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:35.379700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.314737    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:01:35.379700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.320681    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:01:35.462774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.462731    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Nov 03 21:01:35.615676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.615629    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:01:35.631814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.631722    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.659291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:35.632623    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:01:36.347629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.316722    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:36.522256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.522213    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:36.545908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.541523    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:01:36.546204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.546186    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:01:36.553531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.553506    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:01:36.587585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:36.587202    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:01:37.009164 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:37.009127    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:01:37.085526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:37.085495    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 21:01:37.085739 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:37.085715    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy" containerID="cri-o://fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35" gracePeriod=30
Nov 03 21:01:37.240753 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:37.240717    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:39.060320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:39.060286    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"]
Nov 03 21:01:39.063918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:39.063894    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-278f121b-g7gl2"]
Nov 03 21:01:39.682688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:39.682656    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="150bb752-c130-431c-9215-48112c82ed70" path="/var/lib/kubelet/pods/150bb752-c130-431c-9215-48112c82ed70/volumes"
Nov 03 21:01:42.120914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.120884    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:42.443503 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.443477    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 21:01:42.495947 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.495917    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.495951    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.495992    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496069    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496108    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496126    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496147    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496171    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496253    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\") pod \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\" (UID: \"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1\") "
Nov 03 21:01:42.496408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496297    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:42.496408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496311    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:42.496618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496465    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:01:42.496618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496594    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:42.496844 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.496777    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:42.497057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.497001    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.497057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.497038    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.497057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.497052    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.497245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.497115    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.497245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.497131    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.498495 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.498473    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:42.498598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.498581    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "kube-api-access-dx2pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:42.498887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.498874    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:01:42.498965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.498953    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy" pod "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" (UID: "7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:42.570787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.570759    5915 generic.go:356] "Generic (PLEG): container finished" podID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerID="fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35" exitCode=0
Nov 03 21:01:42.570909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.570809    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"
Nov 03 21:01:42.570909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.570851    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" event={"ID":"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1","Type":"ContainerDied","Data":"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"}
Nov 03 21:01:42.570983 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.570907    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk" event={"ID":"7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1","Type":"ContainerDied","Data":"33ee77a82fba6873c7de3e8e428cb09dfa5117f6004754bed06d117706422869"}
Nov 03 21:01:42.570983 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.570937    5915 scope.go:122] "RemoveContainer" containerID="fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"
Nov 03 21:01:42.583938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.583920    5915 scope.go:122] "RemoveContainer" containerID="fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"
Nov 03 21:01:42.584222 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:42.584202    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35\": container with ID starting with fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35 not found: ID does not exist" containerID="fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"
Nov 03 21:01:42.584277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.584231    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35"} err="failed to get container status \"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35\": rpc error: code = NotFound desc = could not find container \"fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35\": container with ID starting with fc774e521c6817b65eff28317d3dbb87e7a929cc70a61286b43d419d472c1b35 not found: ID does not exist"
Nov 03 21:01:42.598012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.597991    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.598129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.598013    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx2pq\" (UniqueName: \"kubernetes.io/projected/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-kube-api-access-dx2pq\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.598129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.598026    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.598129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.598034    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:42.728861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.728828    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 21:01:42.732431 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:42.732401    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-hz2fk"]
Nov 03 21:01:43.682591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:43.682541    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" path="/var/lib/kubelet/pods/7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1/volumes"
Nov 03 21:01:46.970240 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:46.970186    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:51.942861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:51.942829    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:52.207272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.207242    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 21:01:52.207459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.207438    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy" containerID="cri-o://ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab" gracePeriod=30
Nov 03 21:01:52.256330 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.256303    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 21:01:52.256514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.256494    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" containerID="cri-o://7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d" gracePeriod=30
Nov 03 21:01:52.421861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.421831    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:01:52.422730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.422707    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy"
Nov 03 21:01:52.422730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.422727    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy"
Nov 03 21:01:52.422936 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.422882    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bbfe6a1-1a4c-4d07-ac7b-ac71aa4ddfb1" containerName="istio-proxy"
Nov 03 21:01:52.428223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.428198    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.487800 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.487741    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:01:52.549463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549423    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.549463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549466    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.549748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549648    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.549748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549705    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.549863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549835    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.549957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549942    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.550019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.549997    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf6c6\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.550102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.550074    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.550165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.550123    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651160    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651200    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651315    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651448    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651520    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651589    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf6c6\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651613    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651647    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651680    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.651998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.651968    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.652149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.652122    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.652217 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.652154    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.652259 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.652240    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.653749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.653722    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.654291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.654269    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.712052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.712023    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.712231 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.712210    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf6c6\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:52.742325 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:52.742266    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:53.178110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.178077    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:01:53.178495 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:01:53.178379    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb95f81_3b62_456e_9294_2505824fbc81.slice/crio-1295d877e136850a11eeed25c3bcc3cb6980c72355356782dbe908d54eb7c634 WatchSource:0}: Error finding container 1295d877e136850a11eeed25c3bcc3cb6980c72355356782dbe908d54eb7c634: Status 404 returned error can't find the container with id 1295d877e136850a11eeed25c3bcc3cb6980c72355356782dbe908d54eb7c634
Nov 03 21:01:53.180913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.180882    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:53.181021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.180976    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:53.181087 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.181022    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:53.660688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.660618    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" event={"ID":"cbb95f81-3b62-456e-9294-2505824fbc81","Type":"ContainerStarted","Data":"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"}
Nov 03 21:01:53.660922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.660770    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" event={"ID":"cbb95f81-3b62-456e-9294-2505824fbc81","Type":"ContainerStarted","Data":"1295d877e136850a11eeed25c3bcc3cb6980c72355356782dbe908d54eb7c634"}
Nov 03 21:01:53.742933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.742900    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:01:53.832901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:53.832842    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podStartSLOduration=1.832828461 podStartE2EDuration="1.832828461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:53.779847345 +0000 UTC m=+1328.231216523" watchObservedRunningTime="2026-11-03 21:01:53.832828461 +0000 UTC m=+1328.284197637"
Nov 03 21:01:54.744657 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:54.744625    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:54.745129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:54.744696    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:01:55.742830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:55.742769    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:55.742830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:55.742816    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:01:55.826079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:55.826046    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-247zc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:01:55.826494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:55.826114    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:01:56.743377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:56.743330    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:56.743586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:56.743393    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:01:56.780727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:56.780704    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:01:57.591404 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.589825    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 21:01:57.661146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.661118    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 21:01:57.693407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.693373    5915 generic.go:356] "Generic (PLEG): container finished" podID="4d7e72bd-ec36-4625-a884-555f558852c3" containerID="ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab" exitCode=0
Nov 03 21:01:57.693937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.693915    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" event={"ID":"4d7e72bd-ec36-4625-a884-555f558852c3","Type":"ContainerDied","Data":"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"}
Nov 03 21:01:57.694102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.694087    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h" event={"ID":"4d7e72bd-ec36-4625-a884-555f558852c3","Type":"ContainerDied","Data":"5c9a302ddf704476f863559d5ce8c80a527acd97b71223e78e077e415dad93c5"}
Nov 03 21:01:57.694267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.693949    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"
Nov 03 21:01:57.694549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.694134    5915 scope.go:122] "RemoveContainer" containerID="ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"
Nov 03 21:01:57.695623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.695600    5915 generic.go:356] "Generic (PLEG): container finished" podID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerID="7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d" exitCode=0
Nov 03 21:01:57.695723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.695709    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" event={"ID":"c3f1cc95-8bb8-4edf-b784-ded87f169271","Type":"ContainerDied","Data":"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"}
Nov 03 21:01:57.695772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.695749    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc" event={"ID":"c3f1cc95-8bb8-4edf-b784-ded87f169271","Type":"ContainerDied","Data":"bb5c3523034801e2124165d2e94889e5d2f4532ad9a7ae5cc2e91ddd29b7b1df"}
Nov 03 21:01:57.695822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.695810    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"
Nov 03 21:01:57.733857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.733759    5915 scope.go:122] "RemoveContainer" containerID="ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"
Nov 03 21:01:57.734415 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:57.734390    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab\": container with ID starting with ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab not found: ID does not exist" containerID="ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"
Nov 03 21:01:57.734494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.734427    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab"} err="failed to get container status \"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab\": rpc error: code = NotFound desc = could not find container \"ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab\": container with ID starting with ee9eb8122e6bc05abb36b7adc14242423bb5da42f4d5eb27fab06b0c58e7f0ab not found: ID does not exist"
Nov 03 21:01:57.734494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.734451    5915 scope.go:122] "RemoveContainer" containerID="7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"
Nov 03 21:01:57.743084 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.742758    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:57.743084 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.742799    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:01:57.746195 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.746127    5915 scope.go:122] "RemoveContainer" containerID="7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"
Nov 03 21:01:57.746367 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:01:57.746349    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d\": container with ID starting with 7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d not found: ID does not exist" containerID="7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"
Nov 03 21:01:57.746426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.746388    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d"} err="failed to get container status \"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d\": rpc error: code = NotFound desc = could not find container \"7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d\": container with ID starting with 7c26f1cd8ca661ab455dbb7edecf69bc39a35df7c059b4b0e706c62e0b8f427d not found: ID does not exist"
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.756846    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.756908    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.756951    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757038    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757139    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757192    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757231    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757245    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757304    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757379    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757489    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757390    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.757668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757604    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757687    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757717    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757751    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757820    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757862    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.757922    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758008    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758069    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\") pod \"c3f1cc95-8bb8-4edf-b784-ded87f169271\" (UID: \"c3f1cc95-8bb8-4edf-b784-ded87f169271\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758194    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758305    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\") pod \"4d7e72bd-ec36-4625-a884-555f558852c3\" (UID: \"4d7e72bd-ec36-4625-a884-555f558852c3\") "
Nov 03 21:01:57.758605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758539    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.759219 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.758986    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.759219 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.759191    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.759870 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.759584    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.759870 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.759797    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.760612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760532    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3f1cc95-8bb8-4edf-b784-ded87f169271-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760594    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760609    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760623    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760637    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d7e72bd-ec36-4625-a884-555f558852c3-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760682    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760698    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760713    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760729    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.760823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.760743    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.762722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.762694    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:01:57.763427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.763393    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.766981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.766945    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.767196 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.767161    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:01:57.767196 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.767177    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "kube-api-access-w8czp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.767572 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.767524    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.768349 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.768310    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy" pod "c3f1cc95-8bb8-4edf-b784-ded87f169271" (UID: "c3f1cc95-8bb8-4edf-b784-ded87f169271"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.769692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.769670    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm" pod "4d7e72bd-ec36-4625-a884-555f558852c3" (UID: "4d7e72bd-ec36-4625-a884-555f558852c3"). InnerVolumeSpecName "kube-api-access-z8qhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861633    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8qhm\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-kube-api-access-z8qhm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861659    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d7e72bd-ec36-4625-a884-555f558852c3-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861669    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8czp\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-kube-api-access-w8czp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861678    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861688    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d7e72bd-ec36-4625-a884-555f558852c3-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861696    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.862037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861714    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d7e72bd-ec36-4625-a884-555f558852c3-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.862037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:57.861723    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3f1cc95-8bb8-4edf-b784-ded87f169271-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:58.168924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.168838    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 21:01:58.175508 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.175474    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-98f8h"]
Nov 03 21:01:58.284993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.284964    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 21:01:58.293446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.293421    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-247zc"]
Nov 03 21:01:58.743629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.743546    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:58.743629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:58.743619    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:01:59.683014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:59.682986    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" path="/var/lib/kubelet/pods/4d7e72bd-ec36-4625-a884-555f558852c3/volumes"
Nov 03 21:01:59.683481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:59.683467    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" path="/var/lib/kubelet/pods/c3f1cc95-8bb8-4edf-b784-ded87f169271/volumes"
Nov 03 21:01:59.742677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:59.742639    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Nov 03 21:01:59.742849 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:01:59.742704    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Nov 03 21:02:00.747206 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:00.747178    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:02:00.749318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:00.749267    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:02:00.758031 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:00.750336    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:02:01.749161 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:01.749127    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:05.236273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.236237    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"]
Nov 03 21:02:05.236970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.236951    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy"
Nov 03 21:02:05.237032 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.236974    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy"
Nov 03 21:02:05.237032 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.237025    5915 cpu_manager.go:414] "containerMap: removing container" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy"
Nov 03 21:02:05.237101 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.237051    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy"
Nov 03 21:02:05.237216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.237205    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3f1cc95-8bb8-4edf-b784-ded87f169271" containerName="istio-proxy"
Nov 03 21:02:05.237256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.237224    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d7e72bd-ec36-4625-a884-555f558852c3" containerName="istio-proxy"
Nov 03 21:02:05.240785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.240769    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.266218 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.266193    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfg52\" (UniqueName: \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.266328 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.266235    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.266386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.266365    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.296093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.296065    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:02:05.345609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.345579    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:02:05.351092 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.351067    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"]
Nov 03 21:02:05.368384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.368358    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfg52\" (UniqueName: \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.368514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.368408    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.368514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.368456    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.369069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.369046    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.370968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.370949    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.429319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.429294    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfg52\" (UniqueName: \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\") pod \"maas-api-revoke-sub-66836146-jhtkq\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.555266 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.555190    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:05.941014 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:05.940978    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e478dc7_f5be_4029_be72_d44a7cef8211.slice/crio-64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e WatchSource:0}: Error finding container 64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e: Status 404 returned error can't find the container with id 64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e
Nov 03 21:02:05.943916 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:05.943889    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"]
Nov 03 21:02:06.676092 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:06.675993    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:06.816725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:06.816684    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq" event={"ID":"4e478dc7-f5be-4029-be72-d44a7cef8211","Type":"ContainerStarted","Data":"ec05e81dbdf11bfcc6bf3e2c8f12550d2da8816418b95084185f03da5676d70e"}
Nov 03 21:02:06.816911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:06.816740    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq" event={"ID":"4e478dc7-f5be-4029-be72-d44a7cef8211","Type":"ContainerStarted","Data":"64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e"}
Nov 03 21:02:06.973585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:06.966740    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq" podStartSLOduration=1.9667163410000001 podStartE2EDuration="1.966716341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:06.940805867 +0000 UTC m=+1341.392175048" watchObservedRunningTime="2026-11-03 21:02:06.966716341 +0000 UTC m=+1341.418085518"
Nov 03 21:02:07.142009 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:07.139278    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:02:07.874596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:07.872422    5915 generic.go:356] "Generic (PLEG): container finished" podID="4e478dc7-f5be-4029-be72-d44a7cef8211" containerID="ec05e81dbdf11bfcc6bf3e2c8f12550d2da8816418b95084185f03da5676d70e" exitCode=0
Nov 03 21:02:07.874596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:07.872790    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq" event={"ID":"4e478dc7-f5be-4029-be72-d44a7cef8211","Type":"ContainerDied","Data":"ec05e81dbdf11bfcc6bf3e2c8f12550d2da8816418b95084185f03da5676d70e"}
Nov 03 21:02:08.349871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:08.349649    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:08.584779 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:08.584721    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:08.604981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:08.604874    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:08.612812 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:08.612761    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Nov 03 21:02:08.640585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:08.613675    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.375676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.351638    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.407585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.384721    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:02:09.471586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.459683    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.594630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.585661    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.619710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.619660    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.624584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.619960    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.664731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.624842    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:02:09.664731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.658248    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:02:09.664731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.647653    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:02:09.664731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:09.659091    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:02:10.377628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.351944    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.671584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.670514    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.671916 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.671791    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:02:10.687397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.674622    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": context deadline exceeded"
Nov 03 21:02:10.687397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.674691    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.687397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.674902    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.687397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.676269    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:02:10.832011 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:10.831956    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.412587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.375605    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:02:11.584825 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.584792    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:11.748699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.737476    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.748699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.740657    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.754481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.754439    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.754986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.754957    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 03 21:02:11.773984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.756257    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\" (UniqueName: \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\") pod \"4e478dc7-f5be-4029-be72-d44a7cef8211\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") "
Nov 03 21:02:11.799583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.774297    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\") pod \"4e478dc7-f5be-4029-be72-d44a7cef8211\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") "
Nov 03 21:02:11.894120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.894067    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\") pod \"4e478dc7-f5be-4029-be72-d44a7cef8211\" (UID: \"4e478dc7-f5be-4029-be72-d44a7cef8211\") "
Nov 03 21:02:11.909981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.909941    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca" pod "4e478dc7-f5be-4029-be72-d44a7cef8211" (UID: "4e478dc7-f5be-4029-be72-d44a7cef8211"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:02:11.961941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.765662    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.996726 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:11.996686    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca" pod "4e478dc7-f5be-4029-be72-d44a7cef8211" (UID: "4e478dc7-f5be-4029-be72-d44a7cef8211"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:02:12.004970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.004865    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:12.005184 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.005164    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e478dc7-f5be-4029-be72-d44a7cef8211-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:12.141611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.129361    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:12.141611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.129438    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:12.405658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.366805    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52" pod "4e478dc7-f5be-4029-be72-d44a7cef8211" (UID: "4e478dc7-f5be-4029-be72-d44a7cef8211"). InnerVolumeSpecName "kube-api-access-qfg52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:02:12.424090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.424056    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:02:12.430831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.430805    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-d57cc4"]
Nov 03 21:02:12.432052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.432025    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4e478dc7-f5be-4029-be72-d44a7cef8211" containerName="revoke-keys"
Nov 03 21:02:12.432217 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.432202    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e478dc7-f5be-4029-be72-d44a7cef8211" containerName="revoke-keys"
Nov 03 21:02:12.432484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.432468    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e478dc7-f5be-4029-be72-d44a7cef8211" containerName="revoke-keys"
Nov 03 21:02:12.473013 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.471036    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfg52\" (UniqueName: \"kubernetes.io/projected/4e478dc7-f5be-4029-be72-d44a7cef8211-kube-api-access-qfg52\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:12.541085 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.477255    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:12.569094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.569065    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:02:12.571356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.571333    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:02:12.710643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.706699    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:02:12.710643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.609116    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nrr89\""
Nov 03 21:02:12.725589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.725325    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:02:12.874141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.790843    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc4sn\" (UniqueName: \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\") pod \"test-curl-7364-d57cc4\" (UID: \"169c06aa-762b-45c4-a188-b9d4cceced20\") " pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:12.927332 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.927287    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-d57cc4"]
Nov 03 21:02:13.018651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:12.977316    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc4sn\" (UniqueName: \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\") pod \"test-curl-7364-d57cc4\" (UID: \"169c06aa-762b-45c4-a188-b9d4cceced20\") " pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:13.213192 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.213154    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc4sn\" (UniqueName: \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\") pod \"test-curl-7364-d57cc4\" (UID: \"169c06aa-762b-45c4-a188-b9d4cceced20\") " pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:13.236088 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.219607    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq" event={"ID":"4e478dc7-f5be-4029-be72-d44a7cef8211","Type":"ContainerDied","Data":"64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e"}
Nov 03 21:02:13.246710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.246660    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64b8ba7deb87839a7242f8433711062ae9cef3602fe037d0723adf82e941104e"
Nov 03 21:02:13.246952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.227129    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:13.248060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.222294    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"
Nov 03 21:02:13.712355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.712326    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-d57cc4"]
Nov 03 21:02:13.713251 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:13.713117    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod169c06aa_762b_45c4_a188_b9d4cceced20.slice/crio-4d48176714dc72990cd3294f69e1291a82662cfb295938185fd91c1d8a3f4c2e WatchSource:0}: Error finding container 4d48176714dc72990cd3294f69e1291a82662cfb295938185fd91c1d8a3f4c2e: Status 404 returned error can't find the container with id 4d48176714dc72990cd3294f69e1291a82662cfb295938185fd91c1d8a3f4c2e
Nov 03 21:02:13.715717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.715696    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:02:13.792925 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:13.792889    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:14.162639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:14.162506    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:14.241734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:14.241702    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-d57cc4" event={"ID":"169c06aa-762b-45c4-a188-b9d4cceced20","Type":"ContainerStarted","Data":"4d48176714dc72990cd3294f69e1291a82662cfb295938185fd91c1d8a3f4c2e"}
Nov 03 21:02:16.008351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.008317    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"]
Nov 03 21:02:16.018075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.018049    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.071047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.071014    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"]
Nov 03 21:02:16.093404 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.093375    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.093586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.093416    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbpw\" (UniqueName: \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.093772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.093746    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.114644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.114614    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:02:16.114777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.114759    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:02:16.194795 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.194758    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.194969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.194869    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.194969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.194895    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbpw\" (UniqueName: \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.195550 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.195524    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.197693 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.197672    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.263417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.263340    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbpw\" (UniqueName: \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\") pod \"maas-api-revoke-sub-5f0ac546-w4d2l\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.336792 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.336761    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:16.730261 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:16.730232    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9fc69d_3f1b_4576_beec_ad4b70282504.slice/crio-fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10 WatchSource:0}: Error finding container fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10: Status 404 returned error can't find the container with id fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10
Nov 03 21:02:16.730940 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:16.730916    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"]
Nov 03 21:02:17.271324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:17.271289    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-d57cc4" event={"ID":"169c06aa-762b-45c4-a188-b9d4cceced20","Type":"ContainerStarted","Data":"de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831"}
Nov 03 21:02:17.273153 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:17.273121    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l" event={"ID":"3f9fc69d-3f1b-4576-beec-ad4b70282504","Type":"ContainerStarted","Data":"facad53502a73997c58cc9d73011bc589301135176fca1499fa1122ffcb723c9"}
Nov 03 21:02:17.273281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:17.273175    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l" event={"ID":"3f9fc69d-3f1b-4576-beec-ad4b70282504","Type":"ContainerStarted","Data":"fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10"}
Nov 03 21:02:17.503359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:17.503301    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-d57cc4" podStartSLOduration=3.771937317 podStartE2EDuration="6.503283097s" totalImagesPullingTime="2.73134578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:17.383040234 +0000 UTC m=+1351.834409413" watchObservedRunningTime="2026-11-03 21:02:17.503283097 +0000 UTC m=+1351.954652276"
Nov 03 21:02:17.504429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:17.504379    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l" podStartSLOduration=2.504365879 podStartE2EDuration="2.504365879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:17.500743371 +0000 UTC m=+1351.952112551" watchObservedRunningTime="2026-11-03 21:02:17.504365879 +0000 UTC m=+1351.955735057"
Nov 03 21:02:18.282715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:18.282679    5915 generic.go:356] "Generic (PLEG): container finished" podID="3f9fc69d-3f1b-4576-beec-ad4b70282504" containerID="facad53502a73997c58cc9d73011bc589301135176fca1499fa1122ffcb723c9" exitCode=0
Nov 03 21:02:18.283115 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:18.282762    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l" event={"ID":"3f9fc69d-3f1b-4576-beec-ad4b70282504","Type":"ContainerDied","Data":"facad53502a73997c58cc9d73011bc589301135176fca1499fa1122ffcb723c9"}
Nov 03 21:02:18.730624 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:18.730591    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:18.900373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:18.899259    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-d57cc4"]
Nov 03 21:02:18.914117 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:18.914078    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-d57cc4"]
Nov 03 21:02:19.290645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.290599    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-d57cc4" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" containerName="test-curl-7364-d57cc4" containerID="cri-o://de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831" gracePeriod=2
Nov 03 21:02:19.341667 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:19.341634    5915 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-d57cc4\" is forbidden: User \"system:node:ip-10-0-141-248.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-248.us-west-1.compute.internal' and this object" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:19.457857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.457832    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:19.502221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.502192    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:19.546342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.546264    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\") pod \"3f9fc69d-3f1b-4576-beec-ad4b70282504\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") "
Nov 03 21:02:19.546527 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.546510    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\") pod \"3f9fc69d-3f1b-4576-beec-ad4b70282504\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") "
Nov 03 21:02:19.546649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.546638    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\" (UniqueName: \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\") pod \"3f9fc69d-3f1b-4576-beec-ad4b70282504\" (UID: \"3f9fc69d-3f1b-4576-beec-ad4b70282504\") "
Nov 03 21:02:19.546900 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.546877    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca" pod "3f9fc69d-3f1b-4576-beec-ad4b70282504" (UID: "3f9fc69d-3f1b-4576-beec-ad4b70282504"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:02:19.547173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.547160    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:19.548573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.548537    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca" pod "3f9fc69d-3f1b-4576-beec-ad4b70282504" (UID: "3f9fc69d-3f1b-4576-beec-ad4b70282504"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:02:19.548921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.548901    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw" pod "3f9fc69d-3f1b-4576-beec-ad4b70282504" (UID: "3f9fc69d-3f1b-4576-beec-ad4b70282504"). InnerVolumeSpecName "kube-api-access-rwbpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:02:19.634452 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:19.634409    5915 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-d57cc4\" is forbidden: User \"system:node:ip-10-0-141-248.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-141-248.us-west-1.compute.internal' and this object" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:19.648260 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.648232    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwbpw\" (UniqueName: \"kubernetes.io/projected/3f9fc69d-3f1b-4576-beec-ad4b70282504-kube-api-access-rwbpw\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:19.648260 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.648257    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f9fc69d-3f1b-4576-beec-ad4b70282504-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:19.672147 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:19.672122    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:20.284054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:20.283989    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:20.298424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:20.298402    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"
Nov 03 21:02:20.298820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:20.298436    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l" event={"ID":"3f9fc69d-3f1b-4576-beec-ad4b70282504","Type":"ContainerDied","Data":"fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10"}
Nov 03 21:02:20.298820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:20.298483    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcc2185e11eb584fa6c74af2cd05b55d066f2cb06970c747935db306c2026e10"
Nov 03 21:02:21.553675 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.553597    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:02:21.555169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555147    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3f9fc69d-3f1b-4576-beec-ad4b70282504" containerName="revoke-keys"
Nov 03 21:02:21.555282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555170    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f9fc69d-3f1b-4576-beec-ad4b70282504" containerName="revoke-keys"
Nov 03 21:02:21.555282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555203    5915 cpu_manager.go:414] "containerMap: removing container" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" containerName="test-curl-7364-d57cc4"
Nov 03 21:02:21.555282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555208    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" containerName="test-curl-7364-d57cc4"
Nov 03 21:02:21.555409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555307    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" containerName="test-curl-7364-d57cc4"
Nov 03 21:02:21.555409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.555324    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f9fc69d-3f1b-4576-beec-ad4b70282504" containerName="revoke-keys"
Nov 03 21:02:21.559060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.559042    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.559769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.559750    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:21.603156 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.603129    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:02:21.607222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.607204    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.676778 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.676751    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\" (UniqueName: \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\") pod \"169c06aa-762b-45c4-a188-b9d4cceced20\" (UID: \"169c06aa-762b-45c4-a188-b9d4cceced20\") "
Nov 03 21:02:21.677150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677132    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677165    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677329 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677306    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677393 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677372    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-856dr\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677466 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677420    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677526    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677681 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677620    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.677767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.677728    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.678417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.678039    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.679794 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.679758    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn" pod "169c06aa-762b-45c4-a188-b9d4cceced20" (UID: "169c06aa-762b-45c4-a188-b9d4cceced20"). InnerVolumeSpecName "kube-api-access-lc4sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:02:21.684980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.684952    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="169c06aa-762b-45c4-a188-b9d4cceced20" path="/var/lib/kubelet/pods/169c06aa-762b-45c4-a188-b9d4cceced20/volumes"
Nov 03 21:02:21.685496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.685481    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:02:21.736435 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.736405    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:02:21.779742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.779711    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.779931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.779765    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.779931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.779821    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.779931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.779871    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-856dr\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.780213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.780191    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.780583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.780545    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.780857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.779913    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.780922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.780892    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.780968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.780926    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv8mm\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.780968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.780962    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.781103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781083    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.781160 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781129    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.781220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781161    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.781276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781216    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.781276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781245    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.781366 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781292    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.781459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781440    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.781547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781526    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781751    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781879    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.781954    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.782032    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.782145    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc4sn\" (UniqueName: \"kubernetes.io/projected/169c06aa-762b-45c4-a188-b9d4cceced20-kube-api-access-lc4sn\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:21.782635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.782588    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.784371 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.784350    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.784824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.784806    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.841284 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.841218    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.841408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.841356    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-856dr\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.872998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.872972    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:21.883909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.883882    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.883929    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.883969    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884028    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884084    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884136    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv8mm\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884189    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884379 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884224    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884379 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884278    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884379 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884296    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884525 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884380    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884625    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884647    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.884964 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.884947    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.886513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.886487    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.886940 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.886918    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.946887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.946341    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:21.959138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:21.959107    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv8mm\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:22.220885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.220855    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:22.321935 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:22.321901    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7175c94c_c24a_4912_a86f_f0c9b97ac5ec.slice/crio-b43e85209a28e6b4215bcbaaa0afd851508b05bd688d73e4d2a8877444d55258 WatchSource:0}: Error finding container b43e85209a28e6b4215bcbaaa0afd851508b05bd688d73e4d2a8877444d55258: Status 404 returned error can't find the container with id b43e85209a28e6b4215bcbaaa0afd851508b05bd688d73e4d2a8877444d55258
Nov 03 21:02:22.323043 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.323021    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:02:22.326547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.325802    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.326547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.325884    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.326547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.325959    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.327847 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.327818    5915 generic.go:356] "Generic (PLEG): container finished" podID="169c06aa-762b-45c4-a188-b9d4cceced20" containerID="de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831" exitCode=137
Nov 03 21:02:22.327955 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.327932    5915 scope.go:122] "RemoveContainer" containerID="de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831"
Nov 03 21:02:22.328061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.328048    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d57cc4"
Nov 03 21:02:22.351663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.351640    5915 scope.go:122] "RemoveContainer" containerID="de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831"
Nov 03 21:02:22.352156 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:22.352074    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831\": container with ID starting with de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831 not found: ID does not exist" containerID="de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831"
Nov 03 21:02:22.352156 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.352106    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831"} err="failed to get container status \"de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831\": rpc error: code = NotFound desc = could not find container \"de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831\": container with ID starting with de9776b7892c57a53dfa31738edae756309c747e9bc41869a7852983fec11831 not found: ID does not exist"
Nov 03 21:02:22.611928 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.611898    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:02:22.613350 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:22.613310    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37882b8d_ba0a_41b3_b1d6_f8c896248387.slice/crio-4073b58a5e84df4c67002ec16e9769d3c50626d2c2e399e22f61d283c11218e1 WatchSource:0}: Error finding container 4073b58a5e84df4c67002ec16e9769d3c50626d2c2e399e22f61d283c11218e1: Status 404 returned error can't find the container with id 4073b58a5e84df4c67002ec16e9769d3c50626d2c2e399e22f61d283c11218e1
Nov 03 21:02:22.616811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.616774    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.616907 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.616863    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.616949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:22.616910    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:23.359512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.359475    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" event={"ID":"37882b8d-ba0a-41b3-b1d6-f8c896248387","Type":"ContainerStarted","Data":"69632878bcbd13bff2b3a4eae144f7e8df4a7a2e4d766ae61a2ec788d43d83a1"}
Nov 03 21:02:23.359710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.359587    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" event={"ID":"37882b8d-ba0a-41b3-b1d6-f8c896248387","Type":"ContainerStarted","Data":"4073b58a5e84df4c67002ec16e9769d3c50626d2c2e399e22f61d283c11218e1"}
Nov 03 21:02:23.361971 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.361944    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" event={"ID":"7175c94c-c24a-4912-a86f-f0c9b97ac5ec","Type":"ContainerStarted","Data":"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"}
Nov 03 21:02:23.362055 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.361997    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" event={"ID":"7175c94c-c24a-4912-a86f-f0c9b97ac5ec","Type":"ContainerStarted","Data":"b43e85209a28e6b4215bcbaaa0afd851508b05bd688d73e4d2a8877444d55258"}
Nov 03 21:02:23.596063 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.596014    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podStartSLOduration=2.595999952 podStartE2EDuration="2.595999952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:23.549976151 +0000 UTC m=+1358.001345340" watchObservedRunningTime="2026-11-03 21:02:23.595999952 +0000 UTC m=+1358.047369129"
Nov 03 21:02:23.680892 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.680787    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podStartSLOduration=2.68077322 podStartE2EDuration="2.68077322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:23.677442724 +0000 UTC m=+1358.128811902" watchObservedRunningTime="2026-11-03 21:02:23.68077322 +0000 UTC m=+1358.132142396"
Nov 03 21:02:23.691550 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.691527    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:23.897581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.891423    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:23.922583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.898685    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:23.922583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:23.898747    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:24.223710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.223622    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:24.249584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.242702    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:24.249584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.242765    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:24.442584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.438611    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:24.877583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.876009    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:24.877583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:24.876081    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:25.187665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.180236    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:02:25.218581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.218007    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.246102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.246057    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:25.260648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.246342    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.306000    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c704c5a1-openshift-default-dockercfg-tn8qt\""
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327481    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327544    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327634    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327666    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhmxt\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327719    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327762    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327846    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327938    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.338641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.327971    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.365601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.347362    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:25.448751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.448628    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:02:25.665906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.665868    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.666281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.666247    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.820273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.820182    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhmxt\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.820989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.820962    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.826808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.826766    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.827212 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.827184    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.827485 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.827454    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.842825 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.842753    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.843280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.843256    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.843661 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.710760    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:25.843872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.803120    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.859952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.837754    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.862433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.794582    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.878695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.878541    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:25.898622 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.879656    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:25.898622 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.881085    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.900276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.900241    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.900598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.880033    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.904905 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.904879    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:25.958708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:25.958677    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:26.016975 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.016934    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhmxt\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\") pod \"e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") " pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:26.095228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.095145    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:26.257367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.257215    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:26.257367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.257269    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:26.654018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.653940    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:02:26.660244 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:26.660200    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaab7f1d_fdd4_4171_9032_88660df3d519.slice/crio-0a4b309cd8d3d0593b9b0241addd0b1bf4cfa52a29d410a30a1c438bd95c0d0b WatchSource:0}: Error finding container 0a4b309cd8d3d0593b9b0241addd0b1bf4cfa52a29d410a30a1c438bd95c0d0b: Status 404 returned error can't find the container with id 0a4b309cd8d3d0593b9b0241addd0b1bf4cfa52a29d410a30a1c438bd95c0d0b
Nov 03 21:02:26.663016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.662985    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:26.663096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.663059    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:26.663096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.663090    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:26.815281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.815193    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerStarted","Data":"0a4b309cd8d3d0593b9b0241addd0b1bf4cfa52a29d410a30a1c438bd95c0d0b"}
Nov 03 21:02:26.874296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.874268    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:26.874509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:26.874488    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:27.032435 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.032376    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:27.217596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.216121    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:27.228592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.223905    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:27.228592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.223962    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:27.394266 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.394188    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:27.747069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.735241    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:27.861929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.861867    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerStarted","Data":"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"}
Nov 03 21:02:27.873874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.873842    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:27.874079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:27.874061    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:28.056654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.056498    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podStartSLOduration=3.056477992 podStartE2EDuration="3.056477992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:28.043383707 +0000 UTC m=+1362.494752887" watchObservedRunningTime="2026-11-03 21:02:28.056477992 +0000 UTC m=+1362.507847171"
Nov 03 21:02:28.095702 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.095668    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:28.121585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.115926    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:28.225582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.224934    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:28.225582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.225031    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:28.440953 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.390155    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:28.704637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.688586    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:28.902582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.894735    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:28.931815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.931706    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:28.955215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.954986    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:28.960106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.959796    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:02:28.980648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:28.980542    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:29.262655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:29.262524    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:29.263251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:29.263219    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:29.873435 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:29.873400    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 03 21:02:29.873620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:29.873462    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 03 21:02:29.915463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:29.915429    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:30.221831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.221794    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 03 21:02:30.222025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.221854    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 03 21:02:30.644168 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.644003    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:30.992726 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.992686    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:30.993416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.993389    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:30.993875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.993855    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:30.995927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:30.995907    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:02:31.242899 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:31.242804    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:31.301744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:31.301711    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:31.302238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:31.302212    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:31.303072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:31.303051    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:02:31.859142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:31.859104    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:32.047171 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:32.047141    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:34.347367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:34.347333    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:34.525906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:34.525874    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:35.658512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:35.658477    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:37.004213 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:37.004175    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:02:37.775830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:37.775792    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:40.112106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:40.112073    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:40.415644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:40.346473    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:40.816610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:40.816577    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:44.627584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.585666    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.706816 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.706765    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.758864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.758792    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Nov 03 21:02:44.759242 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.758855    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.992761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.992721    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:44.993048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:44.993017    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.559766 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.559706    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:45.560129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.560099    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.625586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.610720    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:02:45.665613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.656764    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.926765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.708693    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.266015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.759913    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.266015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.869695    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.266015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:45.870075    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:46.266015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:46.262055    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.924194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:46.924133    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:47.070943 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.070640    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:47.089017    5915 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.407s"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.193903    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.193981    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.194044    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.194112    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:02:47.208820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:46.956309    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:02:47.249608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.245818    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:02:47.249608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.246253    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:02:47.290585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.193650    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:47.290585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.252046    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.358694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.358652    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 03 21:02:47.484336 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.411438    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.484576 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.411506    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.900930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.900838    5915 scope.go:122] "RemoveContainer" containerID="d34175c33d10be80534aadba86ea307c3ecf786105af5593ecc0bad36f86123e"
Nov 03 21:02:47.901803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:47.901776    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:48.436034 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.435979    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.436829 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.435867    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.437862 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.250228    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.546720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.546665    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Nov 03 21:02:48.698648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.625534    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.701673 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.701620    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Nov 03 21:02:48.745685 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.745634    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:48.847216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.831707    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.847216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:48.832965    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-8pd8w" podUID="4d6c0c29-53e0-41e7-bc60-1f6cd0d7365e" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.132.0.42:6080/livez\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:48.885751 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:02:48.862753    5915 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.131s"
Nov 03 21:02:49.164641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.155179    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:02:49.198585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.183865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:49.424854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.410381    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:49.444594 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.442641    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:02:49.444594 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.442955    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:49.563325 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.506669    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:49.567390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.566382    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:49.789611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:49.789527    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Nov 03 21:02:50.263363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.263332    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:02:50.341085 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.341059    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:02:50.341247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.341223    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.343950 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.341235    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:02:50.343950 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.342335    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:02:50.347758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.347735    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:02:50.347873 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.347809    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:02:50.347873 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.347844    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:02:50.347992 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.347875    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:02:50.348862 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.348841    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.363496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.363469    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.363812 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.363779    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ddp\" (UniqueName: \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.363921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.363831    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.364012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.363995    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5lvp\" (UniqueName: \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.401187 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.401155    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c704c5a1\""
Nov 03 21:02:50.401393 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.401357    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c704c5a1-dockercfg-g9tmh\""
Nov 03 21:02:50.477084 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.477045    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.478176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.478112    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.478305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.478182    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ddp\" (UniqueName: \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.478305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.478259    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.478418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.478385    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5lvp\" (UniqueName: \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.479400 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.479367    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.539538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.539497    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5lvp\" (UniqueName: \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\") pod \"payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") " pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:50.552673 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.552640    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ddp\" (UniqueName: \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\") pod \"payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.676013 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.675909    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:50.684494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:50.684461    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:51.117037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.117008    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:02:51.118442 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:51.118419    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba05b79_4d05_441d_bcdc_6ef0b7cc980b.slice/crio-7b53c1b6d37b219d2942f1275530e82b52317abf52a5e61bd8609f8220afbc00 WatchSource:0}: Error finding container 7b53c1b6d37b219d2942f1275530e82b52317abf52a5e61bd8609f8220afbc00: Status 404 returned error can't find the container with id 7b53c1b6d37b219d2942f1275530e82b52317abf52a5e61bd8609f8220afbc00
Nov 03 21:02:51.284361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.284327    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:02:51.284734 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:02:51.284708    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34527e2f_c348_41ca_b740_e6fc8d3256cc.slice/crio-67374b9e8608ebef9965c8cd6cc5e11b9791484ea750f6252802560a5d292235 WatchSource:0}: Error finding container 67374b9e8608ebef9965c8cd6cc5e11b9791484ea750f6252802560a5d292235: Status 404 returned error can't find the container with id 67374b9e8608ebef9965c8cd6cc5e11b9791484ea750f6252802560a5d292235
Nov 03 21:02:51.348229 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.348198    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" event={"ID":"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b","Type":"ContainerStarted","Data":"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"}
Nov 03 21:02:51.348407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.348259    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" event={"ID":"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b","Type":"ContainerStarted","Data":"7b53c1b6d37b219d2942f1275530e82b52317abf52a5e61bd8609f8220afbc00"}
Nov 03 21:02:51.348407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.348370    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:02:51.350065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.350038    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" event={"ID":"34527e2f-c348-41ca-b740-e6fc8d3256cc","Type":"ContainerStarted","Data":"67374b9e8608ebef9965c8cd6cc5e11b9791484ea750f6252802560a5d292235"}
Nov 03 21:02:51.463326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:51.463274    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" podStartSLOduration=2.463260344 podStartE2EDuration="2.463260344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:51.462402615 +0000 UTC m=+1385.913771782" watchObservedRunningTime="2026-11-03 21:02:51.463260344 +0000 UTC m=+1385.914629521"
Nov 03 21:02:52.360103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:52.360070    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" event={"ID":"34527e2f-c348-41ca-b740-e6fc8d3256cc","Type":"ContainerStarted","Data":"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"}
Nov 03 21:02:52.360625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:52.360151    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:02:52.477328 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:52.477265    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" podStartSLOduration=3.477245479 podStartE2EDuration="3.477245479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:52.476421293 +0000 UTC m=+1386.927790472" watchObservedRunningTime="2026-11-03 21:02:52.477245479 +0000 UTC m=+1386.928614659"
Nov 03 21:02:53.828613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:53.828575    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:55.725580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:55.724855    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:56.526694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:56.526664    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:58.721348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:58.721232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:58.907309 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:58.907247    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:59.443808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.443775    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:02:59.449020 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.449000    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.502983 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.502940    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6f2c17e9-openshift-default-dockercfg-gtnsn\""
Nov 03 21:02:59.512324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512263    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.512640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512615    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.512796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512696    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.512796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512781    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.512917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512817    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slmxs\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.512917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512853    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.513026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512952    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.513026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.512987    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.513129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.513080    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.558814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.558785    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:02:59.587005 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.586977    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:02:59.614231 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614201    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614255    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614286    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slmxs\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614316    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614364    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614395    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614718 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614472    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614718 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614586    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614748    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.614908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.614874    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.615175 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.615148    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.615662 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.615638    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.615774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.615605    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.615774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.615473    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.617462 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.617443    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.617745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.617726    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.724585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.722351    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slmxs\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.746053 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.745986    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\") pod \"e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") " pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.790600 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.784513    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:02:59.838934 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:02:59.838898    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:00.476337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:00.476309    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:03:00.476512 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:03:00.476490    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43272da_02c4_44cc_9d0b_f7f25f0cd91d.slice/crio-28abc272c062b869397a59e0ae5aba290361166b198eb97ad7b63124049075c2 WatchSource:0}: Error finding container 28abc272c062b869397a59e0ae5aba290361166b198eb97ad7b63124049075c2: Status 404 returned error can't find the container with id 28abc272c062b869397a59e0ae5aba290361166b198eb97ad7b63124049075c2
Nov 03 21:03:00.478965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:00.478935    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:00.479050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:00.478998    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:00.479050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:00.479029    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:01.463908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:01.463872    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerStarted","Data":"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"}
Nov 03 21:03:01.464304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:01.463929    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerStarted","Data":"28abc272c062b869397a59e0ae5aba290361166b198eb97ad7b63124049075c2"}
Nov 03 21:03:01.638152 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:01.638102    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podStartSLOduration=2.638081222 podStartE2EDuration="2.638081222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:01.633914752 +0000 UTC m=+1396.085283928" watchObservedRunningTime="2026-11-03 21:03:01.638081222 +0000 UTC m=+1396.089450398"
Nov 03 21:03:01.789063 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:01.788979    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:03:01.834605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:01.830868    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:03:02.363427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:02.363395    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:03:02.472304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:02.472266    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:03:02.473278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:02.473258    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:03:03.250568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:03.250497    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:03.371822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:03.371790    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:03:03.712675 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:03.712634    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:03.936303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:03.936272    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:04.125632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:04.116046    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:06.469293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:06.469264    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:06.645813 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:06.645760    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:07.010950 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:07.010916    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:03:08.307347 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.307310    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:03:08.312402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.312384    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.491748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.491717    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.491920 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.491754    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.491920 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.491772    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.491999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.491949    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.492035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.491995    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.492131 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.492108    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.492214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.492201    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.492269 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.492257    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.492321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.492309    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkc4t\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.525128 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.525100    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:03:08.593646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593535    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.593646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593604    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.593863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593649    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.593863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593696    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.593863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593747    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.593863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.593834    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkc4t\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.594091 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594069    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.594178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594153    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.594234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594177    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.594234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594195    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.595236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594370    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.595236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594506    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.595236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594612    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.595236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.594682    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.596601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.596575    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.596883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.596866    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.654662 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.654630    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.654841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.654704    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkc4t\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:08.928412 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:08.928334    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:09.378513 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:03:09.378485    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6080f9_7776_494d_8292_f542b7497164.slice/crio-df3463362dda92f416352c2029b31b96ec02373d3869f67ae49ddcb586083c70 WatchSource:0}: Error finding container df3463362dda92f416352c2029b31b96ec02373d3869f67ae49ddcb586083c70: Status 404 returned error can't find the container with id df3463362dda92f416352c2029b31b96ec02373d3869f67ae49ddcb586083c70
Nov 03 21:03:09.378886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.378660    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:03:09.381162 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.381133    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.381270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.381207    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.381270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.381254    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.518654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.518616    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:09.544894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.544859    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" event={"ID":"ab6080f9-7776-494d-8292-f542b7497164","Type":"ContainerStarted","Data":"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"}
Nov 03 21:03:09.545066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.544934    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" event={"ID":"ab6080f9-7776-494d-8292-f542b7497164","Type":"ContainerStarted","Data":"df3463362dda92f416352c2029b31b96ec02373d3869f67ae49ddcb586083c70"}
Nov 03 21:03:09.704370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.704255    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podStartSLOduration=1.704230575 podStartE2EDuration="1.704230575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:09.689454748 +0000 UTC m=+1404.140823925" watchObservedRunningTime="2026-11-03 21:03:09.704230575 +0000 UTC m=+1404.155599753"
Nov 03 21:03:09.928666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:09.928633    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:10.128782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:10.128690    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:10.317584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:10.314040    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:10.468785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:10.468758    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:10.931324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:10.931240    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:10.931324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:10.931293    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:11.928899 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:11.928858    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:11.929333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:11.928945    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:12.243924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:12.243894    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:12.535952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:12.535872    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:12.929110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:12.929007    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:12.929674 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:12.929644    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:13.924136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:13.924096    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:13.928989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:13.928963    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:13.929162 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:13.929006    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:14.117629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:14.117293    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:14.824581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:14.824226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:14.935732 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:14.935704    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:14.936192 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:14.936155    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:15.247270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.247237    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:03:15.251986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.251970    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.288063 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.288038    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h988\" (UniqueName: \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.288204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.288086    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.288204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.288172    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.288204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.288199    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.318584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.317462    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c704c5a1-serving-cert\""
Nov 03 21:03:15.375657 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.375626    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:03:15.390433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.390396    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.390581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.390471    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.390581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.390498    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.390897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.390877    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h988\" (UniqueName: \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.391076 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:15.391059    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-serving-cert: secret "maas-api-e2e-ait-c704c5a1-serving-cert" not found
Nov 03 21:03:15.391148 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:15.391135    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls podName:f9119a60-e5be-4d67-8ab4-26680eb7738b nodeName:}" failed. No retries permitted until 2026-11-03 21:03:15.891113613 +0000 UTC m=+1410.342482768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls") pod "maas-api-e2e-ait-c704c5a1-884c887b5-69crj" (UID: "f9119a60-e5be-4d67-8ab4-26680eb7738b") : secret "maas-api-e2e-ait-c704c5a1-serving-cert" not found
Nov 03 21:03:15.391790 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.391763    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.393628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.393609    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.430976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.430943    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:15.462201 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.462171    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h988\" (UniqueName: \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.671334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.671228    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:15.848621 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.848503    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:15.928953 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.928851    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 03 21:03:15.928953 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.928930    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 03 21:03:15.975517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.975486    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:15.978282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:15.978250    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") pod \"maas-api-e2e-ait-c704c5a1-884c887b5-69crj\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:16.167344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:16.167315    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:16.573488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:16.573444    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:03:16.827864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:16.827783    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" event={"ID":"f9119a60-e5be-4d67-8ab4-26680eb7738b","Type":"ContainerStarted","Data":"d28ebf341cca68d91907d8e17680a351aa9b66cf4a54a0f3bc37dc6aa1405406"}
Nov 03 21:03:17.418365 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.418339    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:17.419111 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.419084    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:17.419985 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.419967    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:03:17.837439 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.837406    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" event={"ID":"f9119a60-e5be-4d67-8ab4-26680eb7738b","Type":"ContainerStarted","Data":"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"}
Nov 03 21:03:17.837691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.837673    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:17.958778 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:17.958719    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" podStartSLOduration=1.942863144 podStartE2EDuration="2.958702872s" totalImagesPullingTime="1.015839728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:17.956891217 +0000 UTC m=+1412.408260397" watchObservedRunningTime="2026-11-03 21:03:17.958702872 +0000 UTC m=+1412.410072052"
Nov 03 21:03:20.557099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:20.557065    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:21.308957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:21.308886    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:21.542805 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:21.542723    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:23.851463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:23.851437    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:03:25.267761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.267726    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:03:25.273295 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.273276    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.327082 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.327039    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-46590df2-serving-cert\""
Nov 03 21:03:25.376107 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.376070    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-562z5\" (UniqueName: \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.376289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.376126    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.376395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.376373    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.376580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.376538    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.381148 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.381121    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:03:25.466499 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.466467    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:25.478063 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.478041    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.478188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.478130    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.478235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.478200    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-562z5\" (UniqueName: \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.478235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.478223    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.478314 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:25.478261    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-serving-cert: secret "maas-api-e2e-migrate-46590df2-serving-cert" not found
Nov 03 21:03:25.478368 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:25.478355    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls podName:2056ea49-f5a0-4021-80ed-f5b2092b8656 nodeName:}" failed. No retries permitted until 2026-11-03 21:03:25.978332845 +0000 UTC m=+1420.429702000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls") pod "maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" (UID: "2056ea49-f5a0-4021-80ed-f5b2092b8656") : secret "maas-api-e2e-migrate-46590df2-serving-cert" not found
Nov 03 21:03:25.478844 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.478828    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.480731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.480711    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.539343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.539285    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-562z5\" (UniqueName: \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:25.612270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.612235    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:25.759635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:25.759522    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:26.023201 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:26.023165    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:26.025931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:26.025904    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") pod \"maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:26.187491 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:26.187458    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:26.661878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:26.660897    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:03:26.927525 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:26.927449    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" event={"ID":"2056ea49-f5a0-4021-80ed-f5b2092b8656","Type":"ContainerStarted","Data":"3fdd7ddfd1e803fd2907453ac0f3d66beafff40f13f4b13ab664c58ff99acbb8"}
Nov 03 21:03:27.937540 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:27.937446    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" event={"ID":"2056ea49-f5a0-4021-80ed-f5b2092b8656","Type":"ContainerStarted","Data":"ff970516946268cc9b756ff8e0e4301f27f23c55dfe9c7270dd25dcda5f31a16"}
Nov 03 21:03:27.937540 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:27.937524    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:28.064360 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:28.064310    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" podStartSLOduration=2.136160335 podStartE2EDuration="3.064296714s" totalImagesPullingTime="928.136379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:28.05927043 +0000 UTC m=+1422.510639606" watchObservedRunningTime="2026-11-03 21:03:28.064296714 +0000 UTC m=+1422.515665890"
Nov 03 21:03:28.832898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:28.832844    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:29.360297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:29.360253    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:30.523276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:30.523243    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:32.562587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:32.559750    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:32.835773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:32.835661    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.755210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:33.755167    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.647586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:33.847078    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.647586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:33.847262    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.647586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:33.950696    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.849210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:34.849164    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.069581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.068953    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:03:35.069581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.069076    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:03:35.713584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.648142    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.721728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.714346    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.780801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.714327    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.783114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.714186    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 03 21:03:36.049403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:35.991848    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Nov 03 21:03:36.466642 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.407966    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:03:36.466642 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.458513    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:03:36.701699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.462643    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:03:36.804636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.797009    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:03:36.804917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.804869    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": context deadline exceeded"
Nov 03 21:03:36.875768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.792372    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:36.990913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.990883    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:03:36.991205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:36.991189    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:03:37.021884 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:37.021856    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:03:37.066052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:37.065926    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:03:37.076666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:37.076640    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:03:37.168452 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:37.168424    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:03:38.158131 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:38.158101    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:41.531423 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:41.531384    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:41.677450 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:41.677416    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:43.003475 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:43.003444    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:44.994699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:44.994667    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:03:44.995250 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:44.994803    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:46.142317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.142232    5915 generic.go:356] "Generic (PLEG): container finished" podID="baab7f1d-fdd4-4171-9032-88660df3d519" containerID="ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b" exitCode=0
Nov 03 21:03:46.142317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.142281    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerDied","Data":"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"}
Nov 03 21:03:46.142878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.142841    5915 scope.go:122] "RemoveContainer" containerID="ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"
Nov 03 21:03:46.145066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.145017    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:46.145179 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.145093    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:46.145179 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:46.145125    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:47.151478 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:47.151446    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerStarted","Data":"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"}
Nov 03 21:03:47.843545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:47.843514    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:48.095742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:48.095661    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:03:48.095901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:48.095823    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 03 21:03:48.095901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:48.095875    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 03 21:03:49.095838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:49.095797    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 03 21:03:49.096354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:49.095849    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 03 21:03:50.096221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:50.096185    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 03 21:03:50.096651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:50.096242    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 03 21:03:51.096205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:51.096167    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 03 21:03:51.096418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:51.096226    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 03 21:03:52.096387 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:52.096349    5915 patch_prober.go:29] interesting pod/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 03 21:03:52.096578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:52.096413    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 03 21:03:52.783429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:52.783400    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:53.100380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.100298    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:03:53.100894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.100802    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:03:53.101708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.101687    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:03:53.235257 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.235222    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"]
Nov 03 21:03:53.238747 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.238725    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.302901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.302872    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.302901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.302905    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303198 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303171    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303238    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303335    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303485 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303466    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303551 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303526    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxz74\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-kube-api-access-cxz74\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303584    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.303630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.303614    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.385159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.385070    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"]
Nov 03 21:03:53.404869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.404825    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.404907    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxz74\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-kube-api-access-cxz74\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.404953    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.404995    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405250 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405107    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405250 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405139    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405272    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405279    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.405373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405306    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.406377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.406377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405622    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.406377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.405875    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.406377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.406255    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.406666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.406392    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.409966 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.409942    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.417598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.412280    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.466349 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.466321    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.466523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.466390    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxz74\" (UniqueName: \"kubernetes.io/projected/d48e0d98-751a-41ff-b3ca-a80292e6c9f4-kube-api-access-cxz74\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z\" (UID: \"d48e0d98-751a-41ff-b3ca-a80292e6c9f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.552377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.552341    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:53.988354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.988327    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"]
Nov 03 21:03:53.989450 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:03:53.989426    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48e0d98_751a_41ff_b3ca_a80292e6c9f4.slice/crio-dc65e7a47d9a78df0c6cfe4ff9fc5ea074b89a632471908cf3df60e9c07278b9 WatchSource:0}: Error finding container dc65e7a47d9a78df0c6cfe4ff9fc5ea074b89a632471908cf3df60e9c07278b9: Status 404 returned error can't find the container with id dc65e7a47d9a78df0c6cfe4ff9fc5ea074b89a632471908cf3df60e9c07278b9
Nov 03 21:03:53.992044 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.992012    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:53.992110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.992081    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:53.992146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:53.992113    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:54.209098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:54.209058    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" event={"ID":"d48e0d98-751a-41ff-b3ca-a80292e6c9f4","Type":"ContainerStarted","Data":"be18b29a67ff0a515c4ace773ff937f2483379b2c44123c246e056d572442ed2"}
Nov 03 21:03:54.209480 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:54.209120    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" event={"ID":"d48e0d98-751a-41ff-b3ca-a80292e6c9f4","Type":"ContainerStarted","Data":"dc65e7a47d9a78df0c6cfe4ff9fc5ea074b89a632471908cf3df60e9c07278b9"}
Nov 03 21:03:54.331017 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:54.330908    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podStartSLOduration=1.330895061 podStartE2EDuration="1.330895061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:54.328470844 +0000 UTC m=+1448.779840022" watchObservedRunningTime="2026-11-03 21:03:54.330895061 +0000 UTC m=+1448.782264232"
Nov 03 21:03:54.553424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:54.553389    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:03:55.554886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:55.554805    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:03:55.554886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:55.554861    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:03:56.552840 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:56.552805    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:03:56.553021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:56.552859    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:03:57.427490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.427454    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:03:57.430849 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.430831    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.473353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.473324    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:03:57.522461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.522427    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6f2c17e9-serving-cert\""
Nov 03 21:03:57.553089 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.553056    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:03:57.553265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.553109    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:03:57.567996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.567943    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6hh\" (UniqueName: \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.568144 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.568085    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.568343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.568316    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.568454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.568438    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.591144 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.591112    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.672810    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.672936    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6hh\" (UniqueName: \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.673052    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.673252    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:57.673443    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-serving-cert: secret "maas-api-e2e-derive-6f2c17e9-serving-cert" not found
Nov 03 21:03:57.673586 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:03:57.673551    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls podName:3db6fb1a-b1e9-49c7-af6b-f2ad071339ef nodeName:}" failed. No retries permitted until 2026-11-03 21:03:58.173527348 +0000 UTC m=+1452.624896526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls") pod "maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" (UID: "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef") : secret "maas-api-e2e-derive-6f2c17e9-serving-cert" not found
Nov 03 21:03:57.704636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.694446    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.710589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.709687    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:57.787612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:57.784324    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6hh\" (UniqueName: \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:58.195874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.195785    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:58.198490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.198461    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") pod \"maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:58.344529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.344499    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:03:58.552715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.552682    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:03:58.553271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.552790    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:03:58.763383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.763357    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:03:58.763593 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:03:58.763534    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db6fb1a_b1e9_49c7_af6b_f2ad071339ef.slice/crio-51343a14b833d207b90b01874c435194bc20b060379a94ab54d9ad5fb112edb8 WatchSource:0}: Error finding container 51343a14b833d207b90b01874c435194bc20b060379a94ab54d9ad5fb112edb8: Status 404 returned error can't find the container with id 51343a14b833d207b90b01874c435194bc20b060379a94ab54d9ad5fb112edb8
Nov 03 21:03:58.847103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:58.847015    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:03:59.262248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:59.262214    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" event={"ID":"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef","Type":"ContainerStarted","Data":"51343a14b833d207b90b01874c435194bc20b060379a94ab54d9ad5fb112edb8"}
Nov 03 21:03:59.553145 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:59.553112    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:03:59.553581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:03:59.553175    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:04:00.273421 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:00.273380    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" event={"ID":"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef","Type":"ContainerStarted","Data":"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"}
Nov 03 21:04:00.274076 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:00.273448    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:04:00.394956 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:00.394904    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" podStartSLOduration=2.606777417 podStartE2EDuration="3.394891303s" totalImagesPullingTime="788.113886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:04:00.390918025 +0000 UTC m=+1454.842287225" watchObservedRunningTime="2026-11-03 21:04:00.394891303 +0000 UTC m=+1454.846260480"
Nov 03 21:04:00.552912 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:00.552823    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:04:00.553667 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:00.553169    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:04:01.553477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:01.553443    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 03 21:04:01.553693 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:01.553497    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 03 21:04:02.690983 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:02.690945    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:04:02.691431 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:02.691409    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:04:02.692439 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:02.692418    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z"
Nov 03 21:04:02.806230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:02.806203    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:06.288289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:06.288256    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:04:07.010667 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:07.010625    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:04:07.464090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.464054    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:04:07.467494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.467471    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.522645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.522611    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e3917193-serving-cert\""
Nov 03 21:04:07.551104 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.551069    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.551317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.551300    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67jrg\" (UniqueName: \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.551398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.551362    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.551533 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.551508    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.576945 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.576920    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:04:07.652766 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.652733    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.652946 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:07.652897    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-serving-cert: secret "maas-api-e2e-mt-e3917193-serving-cert" not found
Nov 03 21:04:07.652946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.652910    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67jrg\" (UniqueName: \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.653069 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:07.652967    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls podName:5318bf4b-8fc2-42c9-b540-35e2ac66ce98 nodeName:}" failed. No retries permitted until 2026-11-03 21:04:08.152946311 +0000 UTC m=+1462.604315482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls") pod "maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" (UID: "5318bf4b-8fc2-42c9-b540-35e2ac66ce98") : secret "maas-api-e2e-mt-e3917193-serving-cert" not found
Nov 03 21:04:07.653069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.652999    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.653177 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.653077    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.653691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.653668    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.655534 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.655513    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.657856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.657815    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:07.714998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.714930    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67jrg\" (UniqueName: \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:07.814427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.814386    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:07.986538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:07.986419    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:08.169619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.169542    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:08.172094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.172071    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") pod \"maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:08.362148 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.362049    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"]
Nov 03 21:04:08.365606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.365552    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.381146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.381123    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:08.475891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.475861    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.475956    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.475993    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476142    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jkbj\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-kube-api-access-5jkbj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476176    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476201    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476227    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d76e840-1ed9-42ef-b9d1-2c889b992835-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476259    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.476326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.476287    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.477756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.477731    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"]
Nov 03 21:04:08.578030 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.577986    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578035    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d76e840-1ed9-42ef-b9d1-2c889b992835-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578097    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578136    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578182    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578464 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578283    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578464 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578453    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578498    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578648    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578661    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578728    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jkbj\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-kube-api-access-5jkbj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578767    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.578978 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.578953    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.579363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.579341    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d76e840-1ed9-42ef-b9d1-2c889b992835-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.581129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.581110    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.581235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.581219    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.639378 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.639318    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.639542 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.639523    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jkbj\" (UniqueName: \"kubernetes.io/projected/4d76e840-1ed9-42ef-b9d1-2c889b992835-kube-api-access-5jkbj\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-xxsms\" (UID: \"4d76e840-1ed9-42ef-b9d1-2c889b992835\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.679744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.679720    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:08.780305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:08.780275    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:04:08.783091 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:04:08.783048    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5318bf4b_8fc2_42c9_b540_35e2ac66ce98.slice/crio-c81557af473667be5a2f3119088cce588e2c883f6e44cc2f91bc4c00f1cc1412 WatchSource:0}: Error finding container c81557af473667be5a2f3119088cce588e2c883f6e44cc2f91bc4c00f1cc1412: Status 404 returned error can't find the container with id c81557af473667be5a2f3119088cce588e2c883f6e44cc2f91bc4c00f1cc1412
Nov 03 21:04:09.068543 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:04:09.068514    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d76e840_1ed9_42ef_b9d1_2c889b992835.slice/crio-fe9ef656a8e18367f1b5c4a03b8f5e6bce0aa9f5ba65eeafac1b81d2f4c53a47 WatchSource:0}: Error finding container fe9ef656a8e18367f1b5c4a03b8f5e6bce0aa9f5ba65eeafac1b81d2f4c53a47: Status 404 returned error can't find the container with id fe9ef656a8e18367f1b5c4a03b8f5e6bce0aa9f5ba65eeafac1b81d2f4c53a47
Nov 03 21:04:09.069628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.069604    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"]
Nov 03 21:04:09.071627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.071551    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.071729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.071658    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.071729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.071688    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.293027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.292983    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:09.348830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.348746    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" event={"ID":"5318bf4b-8fc2-42c9-b540-35e2ac66ce98","Type":"ContainerStarted","Data":"c81557af473667be5a2f3119088cce588e2c883f6e44cc2f91bc4c00f1cc1412"}
Nov 03 21:04:09.351199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.351169    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" event={"ID":"4d76e840-1ed9-42ef-b9d1-2c889b992835","Type":"ContainerStarted","Data":"6c8ae6f09e63c03278314a1f56456ea751afc2e99823423eabb5e7e26a32a4b9"}
Nov 03 21:04:09.351343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.351217    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" event={"ID":"4d76e840-1ed9-42ef-b9d1-2c889b992835","Type":"ContainerStarted","Data":"fe9ef656a8e18367f1b5c4a03b8f5e6bce0aa9f5ba65eeafac1b81d2f4c53a47"}
Nov 03 21:04:09.504929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.504866    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podStartSLOduration=1.504851731 podStartE2EDuration="1.504851731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:04:09.50114929 +0000 UTC m=+1463.952518470" watchObservedRunningTime="2026-11-03 21:04:09.504851731 +0000 UTC m=+1463.956220965"
Nov 03 21:04:09.691056 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:09.690967    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:10.537584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:10.529358    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" event={"ID":"5318bf4b-8fc2-42c9-b540-35e2ac66ce98","Type":"ContainerStarted","Data":"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"}
Nov 03 21:04:10.550010 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:10.549933    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:10.720311 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:10.720240    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" podStartSLOduration=2.9191100800000003 podStartE2EDuration="3.720219132s" totalImagesPullingTime="801.109052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:04:10.688040327 +0000 UTC m=+1465.139409509" watchObservedRunningTime="2026-11-03 21:04:10.720219132 +0000 UTC m=+1465.171588310"
Nov 03 21:04:10.723990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:10.721061    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:10.724245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:10.724216    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:11.125582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.102648    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 03 21:04:11.701645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.696019    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:11.701645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.696081    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:11.751588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.746949    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:11.812588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.809834    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:11.891426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.887717    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:11.961810 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:11.961710    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:12.077363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.076541    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:12.106658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.104667    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:04:12.106658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.105168    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:04:12.346178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.338601    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:04:12.681108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.681021    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:12.681108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:12.681080    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:13.310589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:13.307811    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:13.680919 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:13.680839    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:13.680919 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:13.680885    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:14.739596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:14.739336    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:14.739596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:14.739409    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:15.249204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:15.249171    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:15.680593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:15.680495    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:15.680593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:15.680565    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:16.680405 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:16.680370    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:16.680866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:16.680423    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:17.592271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:17.592239    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:04:17.680008 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:17.679969    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Nov 03 21:04:17.680182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:17.680021    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Nov 03 21:04:18.130020 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:18.129978    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:19.072671 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:19.072635    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:19.073076 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:19.073051    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:19.075014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:19.074994    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms"
Nov 03 21:04:23.279635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:23.279598    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:26.699136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:26.699102    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:26.852430 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:26.852398    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:28.579497 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:28.579464    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:29.156988 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:29.156957    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:29.719179 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:29.719150    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:33.664489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:33.664454    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:37.005527 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:37.005490    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:04:37.606737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:37.606702    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:38.818142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:38.818110    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:40.470807 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:40.470764    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:04:40.586058 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:40.586030    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:04:40.842353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:40.842277    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:41.018626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:41.018581    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:44.556805 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:44.556772    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:45.471529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.471469    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" podUID="34527e2f-c348-41ca-b740-e6fc8d3256cc" containerName="payload-processing" containerID="cri-o://03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621" gracePeriod=25
Nov 03 21:04:45.586463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.586402    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" podUID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" containerName="payload-pre-processing" containerID="cri-o://a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90" gracePeriod=25
Nov 03 21:04:45.874769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.874657    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:04:45.966879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.942241    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\") pod \"34527e2f-c348-41ca-b740-e6fc8d3256cc\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") "
Nov 03 21:04:45.966879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.942436    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\" (UniqueName: \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\") pod \"34527e2f-c348-41ca-b740-e6fc8d3256cc\" (UID: \"34527e2f-c348-41ca-b740-e6fc8d3256cc\") "
Nov 03 21:04:45.966879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.944829    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume" pod "34527e2f-c348-41ca-b740-e6fc8d3256cc" (UID: "34527e2f-c348-41ca-b740-e6fc8d3256cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:04:45.966879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:45.960076    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp" pod "34527e2f-c348-41ca-b740-e6fc8d3256cc" (UID: "34527e2f-c348-41ca-b740-e6fc8d3256cc"). InnerVolumeSpecName "kube-api-access-w5lvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:04:46.040483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.040349    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:04:46.045374 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.045347    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34527e2f-c348-41ca-b740-e6fc8d3256cc-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:04:46.045627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.045607    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5lvp\" (UniqueName: \"kubernetes.io/projected/34527e2f-c348-41ca-b740-e6fc8d3256cc-kube-api-access-w5lvp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:04:46.057599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.057575    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:04:46.063612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.063590    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:04:46.071675 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.071657    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:04:46.076234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.076212    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:04:46.146745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.146713    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\") pod \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") "
Nov 03 21:04:46.147023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.147005    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\" (UniqueName: \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\") pod \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\" (UID: \"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b\") "
Nov 03 21:04:46.147119 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.147067    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume" pod "3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" (UID: "3ba05b79-4d05-441d-bcdc-6ef0b7cc980b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:04:46.147497 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.147478    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:04:46.149398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.149380    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp" pod "3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" (UID: "3ba05b79-4d05-441d-bcdc-6ef0b7cc980b"). InnerVolumeSpecName "kube-api-access-m5ddp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:04:46.234268 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.234235    5915 generic.go:356] "Generic (PLEG): container finished" podID="34527e2f-c348-41ca-b740-e6fc8d3256cc" containerID="03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621" exitCode=0
Nov 03 21:04:46.234580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.234283    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"
Nov 03 21:04:46.234701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.234303    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" event={"ID":"34527e2f-c348-41ca-b740-e6fc8d3256cc","Type":"ContainerDied","Data":"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"}
Nov 03 21:04:46.234701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.234691    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn" event={"ID":"34527e2f-c348-41ca-b740-e6fc8d3256cc","Type":"ContainerDied","Data":"67374b9e8608ebef9965c8cd6cc5e11b9791484ea750f6252802560a5d292235"}
Nov 03 21:04:46.234824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.234735    5915 scope.go:122] "RemoveContainer" containerID="03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"
Nov 03 21:04:46.237035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.237012    5915 generic.go:356] "Generic (PLEG): container finished" podID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" containerID="a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90" exitCode=0
Nov 03 21:04:46.237181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.237057    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"
Nov 03 21:04:46.237345 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.237073    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" event={"ID":"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b","Type":"ContainerDied","Data":"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"}
Nov 03 21:04:46.237422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.237377    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q" event={"ID":"3ba05b79-4d05-441d-bcdc-6ef0b7cc980b","Type":"ContainerDied","Data":"7b53c1b6d37b219d2942f1275530e82b52317abf52a5e61bd8609f8220afbc00"}
Nov 03 21:04:46.248897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.248863    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ddp\" (UniqueName: \"kubernetes.io/projected/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b-kube-api-access-m5ddp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:04:46.259253 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.259129    5915 scope.go:122] "RemoveContainer" containerID="03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"
Nov 03 21:04:46.259632 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:46.259604    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621\": container with ID starting with 03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621 not found: ID does not exist" containerID="03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"
Nov 03 21:04:46.259767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.259741    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621"} err="failed to get container status \"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621\": rpc error: code = NotFound desc = could not find container \"03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621\": container with ID starting with 03bcca5fba4201981d24a92776c2dde505629e2e1f8baf3e50f2528065be7621 not found: ID does not exist"
Nov 03 21:04:46.259880 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.259866    5915 scope.go:122] "RemoveContainer" containerID="a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"
Nov 03 21:04:46.270725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.270704    5915 scope.go:122] "RemoveContainer" containerID="a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"
Nov 03 21:04:46.270983 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:04:46.270965    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90\": container with ID starting with a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90 not found: ID does not exist" containerID="a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"
Nov 03 21:04:46.271048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.270992    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90"} err="failed to get container status \"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90\": rpc error: code = NotFound desc = could not find container \"a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90\": container with ID starting with a1737fdd6eecf68a186fbff972001a1ad463c82ddb32d9b9544c841318341a90 not found: ID does not exist"
Nov 03 21:04:46.428186 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.428098    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:04:46.433045 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.433009    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-c704c5a1-5f78f49b67-x265q"]
Nov 03 21:04:46.535647 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.535616    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:04:46.549451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:46.549424    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c704c5a1-5d7745bc9d-p2pzn"]
Nov 03 21:04:47.526279 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:47.526203    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:47.762584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:47.738216    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:48.072857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:48.072828    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34527e2f-c348-41ca-b740-e6fc8d3256cc" path="/var/lib/kubelet/pods/34527e2f-c348-41ca-b740-e6fc8d3256cc/volumes"
Nov 03 21:04:48.073509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:48.073492    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" path="/var/lib/kubelet/pods/3ba05b79-4d05-441d-bcdc-6ef0b7cc980b/volumes"
Nov 03 21:04:50.432966 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:50.432931    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:55.337712 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.337676    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:04:55.445444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.445376    5915 generic.go:356] "Generic (PLEG): container finished" podID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerID="edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795" exitCode=0
Nov 03 21:04:55.445851 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.445824    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerDied","Data":"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"}
Nov 03 21:04:55.446552 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.446499    5915 scope.go:122] "RemoveContainer" containerID="edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"
Nov 03 21:04:55.449986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.449917    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:55.450834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.450745    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:55.451043 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:55.451024    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:56.458777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:56.458737    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerStarted","Data":"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"}
Nov 03 21:04:56.786711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:56.786618    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:04:56.786879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:56.786809    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:04:56.786879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:56.786852    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:04:57.786243 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:57.786203    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:04:57.786658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:57.786256    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:04:58.786397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:58.786358    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:04:58.786910 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:58.786420    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:04:59.786711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:59.786674    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:04:59.787114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:04:59.786736    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:05:00.253045 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:00.253007    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:00.786814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:00.786780    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:05:00.787191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:00.786839    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:05:01.787021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:01.786985    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 03 21:05:01.787450 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:01.787046    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 03 21:05:02.472884 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:02.472852    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:05:02.791358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:02.791289    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:05:02.792391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:02.792366    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:05:05.318746 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:05.318709    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:07.006099 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:07.006059    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:05:08.057839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:08.057798    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"]
Nov 03 21:05:08.107513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:08.107481    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3438d597-274ln"]
Nov 03 21:05:09.683746 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:09.683703    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e662501-ccd6-4517-88fe-0b078fb2b206" path="/var/lib/kubelet/pods/6e662501-ccd6-4517-88fe-0b078fb2b206/volumes"
Nov 03 21:05:10.446182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:10.446145    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:15.552808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:15.552776    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:15.726017 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:15.725971    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:17.498874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:17.498836    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:17.711658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:17.688466    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:17.996075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:17.996044    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:18.921390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:18.921357    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:19.087933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:19.087864    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:19.573093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:19.573054    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:20.320436 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.320398    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:05:20.320960 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.320655    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" podUID="f9119a60-e5be-4d67-8ab4-26680eb7738b" containerName="maas-api" containerID="cri-o://a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f" gracePeriod=30
Nov 03 21:05:20.470125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.470091    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:20.738864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.738833    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:05:20.819100 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.819059    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") pod \"f9119a60-e5be-4d67-8ab4-26680eb7738b\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") "
Nov 03 21:05:20.819436 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.819416    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\") pod \"f9119a60-e5be-4d67-8ab4-26680eb7738b\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") "
Nov 03 21:05:20.819891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.819862    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\" (UniqueName: \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\") pod \"f9119a60-e5be-4d67-8ab4-26680eb7738b\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") "
Nov 03 21:05:20.820202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.820186    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\") pod \"f9119a60-e5be-4d67-8ab4-26680eb7738b\" (UID: \"f9119a60-e5be-4d67-8ab4-26680eb7738b\") "
Nov 03 21:05:20.824202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.824169    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca" pod "f9119a60-e5be-4d67-8ab4-26680eb7738b" (UID: "f9119a60-e5be-4d67-8ab4-26680eb7738b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:20.873631 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.870729    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls" pod "f9119a60-e5be-4d67-8ab4-26680eb7738b" (UID: "f9119a60-e5be-4d67-8ab4-26680eb7738b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:20.894590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.886888    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988" pod "f9119a60-e5be-4d67-8ab4-26680eb7738b" (UID: "f9119a60-e5be-4d67-8ab4-26680eb7738b"). InnerVolumeSpecName "kube-api-access-4h988". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:20.894590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.894514    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs" pod "f9119a60-e5be-4d67-8ab4-26680eb7738b" (UID: "f9119a60-e5be-4d67-8ab4-26680eb7738b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:20.947444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.947408    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h988\" (UniqueName: \"kubernetes.io/projected/f9119a60-e5be-4d67-8ab4-26680eb7738b-kube-api-access-4h988\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:20.955456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.955428    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:20.955722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.955702    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f9119a60-e5be-4d67-8ab4-26680eb7738b-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:20.955846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.955832    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f9119a60-e5be-4d67-8ab4-26680eb7738b-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:20.973666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.959087    5915 generic.go:356] "Generic (PLEG): container finished" podID="f9119a60-e5be-4d67-8ab4-26680eb7738b" containerID="a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f" exitCode=0
Nov 03 21:05:20.973666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.959324    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" event={"ID":"f9119a60-e5be-4d67-8ab4-26680eb7738b","Type":"ContainerDied","Data":"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"}
Nov 03 21:05:20.973666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.959381    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj" event={"ID":"f9119a60-e5be-4d67-8ab4-26680eb7738b","Type":"ContainerDied","Data":"d28ebf341cca68d91907d8e17680a351aa9b66cf4a54a0f3bc37dc6aa1405406"}
Nov 03 21:05:20.973666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.959412    5915 scope.go:122] "RemoveContainer" containerID="a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"
Nov 03 21:05:20.986602 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:20.974346    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"
Nov 03 21:05:21.048489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:21.048171    5915 scope.go:122] "RemoveContainer" containerID="a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"
Nov 03 21:05:21.049326 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:21.049221    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f\": container with ID starting with a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f not found: ID does not exist" containerID="a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"
Nov 03 21:05:21.049326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:21.049260    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f"} err="failed to get container status \"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f\": rpc error: code = NotFound desc = could not find container \"a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f\": container with ID starting with a6a8341ae8420cc9d5e9722ccfe8b99486d76afb5508e3051f7c3726b005420f not found: ID does not exist"
Nov 03 21:05:21.260923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:21.260855    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:05:21.265114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:21.265085    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c704c5a1-884c887b5-69crj"]
Nov 03 21:05:21.766963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:21.766929    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9119a60-e5be-4d67-8ab4-26680eb7738b" path="/var/lib/kubelet/pods/f9119a60-e5be-4d67-8ab4-26680eb7738b/volumes"
Nov 03 21:05:22.452035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:22.452003    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:22.615274 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:22.615241    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:22.829226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:22.829143    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:23.254203 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:23.254169    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:05:23.254453 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:23.254429    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy" containerID="cri-o://bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a" gracePeriod=30
Nov 03 21:05:23.273972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:23.273941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:23.892195 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:23.892162    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:26.045658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:26.045585    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:26.248429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:26.248388    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:26.474761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:26.474728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:27.306589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:27.296641    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:05:27.307617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:27.307579    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" podUID="2056ea49-f5a0-4021-80ed-f5b2092b8656" containerName="maas-api" containerID="cri-o://ff970516946268cc9b756ff8e0e4301f27f23c55dfe9c7270dd25dcda5f31a16" gracePeriod=30
Nov 03 21:05:27.457954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:27.457927    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:28.056743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.056714    5915 generic.go:356] "Generic (PLEG): container finished" podID="2056ea49-f5a0-4021-80ed-f5b2092b8656" containerID="ff970516946268cc9b756ff8e0e4301f27f23c55dfe9c7270dd25dcda5f31a16" exitCode=0
Nov 03 21:05:28.057134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.057110    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" event={"ID":"2056ea49-f5a0-4021-80ed-f5b2092b8656","Type":"ContainerDied","Data":"ff970516946268cc9b756ff8e0e4301f27f23c55dfe9c7270dd25dcda5f31a16"}
Nov 03 21:05:28.057321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.057301    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq" event={"ID":"2056ea49-f5a0-4021-80ed-f5b2092b8656","Type":"ContainerDied","Data":"3fdd7ddfd1e803fd2907453ac0f3d66beafff40f13f4b13ab664c58ff99acbb8"}
Nov 03 21:05:28.057442 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.057428    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fdd7ddfd1e803fd2907453ac0f3d66beafff40f13f4b13ab664c58ff99acbb8"
Nov 03 21:05:28.072857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.072836    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:05:28.151425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.151363    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\" (UniqueName: \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\") pod \"2056ea49-f5a0-4021-80ed-f5b2092b8656\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") "
Nov 03 21:05:28.151425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.151418    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") pod \"2056ea49-f5a0-4021-80ed-f5b2092b8656\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") "
Nov 03 21:05:28.151655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.151599    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\") pod \"2056ea49-f5a0-4021-80ed-f5b2092b8656\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") "
Nov 03 21:05:28.151724 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.151680    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\") pod \"2056ea49-f5a0-4021-80ed-f5b2092b8656\" (UID: \"2056ea49-f5a0-4021-80ed-f5b2092b8656\") "
Nov 03 21:05:28.151976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.151945    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca" pod "2056ea49-f5a0-4021-80ed-f5b2092b8656" (UID: "2056ea49-f5a0-4021-80ed-f5b2092b8656"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:28.152419 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.152400    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2056ea49-f5a0-4021-80ed-f5b2092b8656-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.153771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.153746    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls" pod "2056ea49-f5a0-4021-80ed-f5b2092b8656" (UID: "2056ea49-f5a0-4021-80ed-f5b2092b8656"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:28.153920 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.153903    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5" pod "2056ea49-f5a0-4021-80ed-f5b2092b8656" (UID: "2056ea49-f5a0-4021-80ed-f5b2092b8656"). InnerVolumeSpecName "kube-api-access-562z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:28.154103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.154066    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs" pod "2056ea49-f5a0-4021-80ed-f5b2092b8656" (UID: "2056ea49-f5a0-4021-80ed-f5b2092b8656"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:28.254318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.254286    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.254318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.254314    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-562z5\" (UniqueName: \"kubernetes.io/projected/2056ea49-f5a0-4021-80ed-f5b2092b8656-kube-api-access-562z5\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.254318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.254326    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2056ea49-f5a0-4021-80ed-f5b2092b8656-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.563376 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.563351    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:05:28.660497 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660468    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.660715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660507    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.660715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660603    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.660715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660644    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.660715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660714    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.660948 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660823    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:28.660948 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660832    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.661057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.660968    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.661142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661105    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.661197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661153    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\") pod \"baab7f1d-fdd4-4171-9032-88660df3d519\" (UID: \"baab7f1d-fdd4-4171-9032-88660df3d519\") "
Nov 03 21:05:28.661197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661177    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:28.661303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661197    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:28.661506 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661484    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:28.661637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.661530    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:28.662260 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.662237    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.662326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.662268    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.662326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.662286    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.662326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.662301    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baab7f1d-fdd4-4171-9032-88660df3d519-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.662326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.662314    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.663536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.663508    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:05:28.663653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.663544    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "kube-api-access-mhmxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:28.663704 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.663670    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:28.663704 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.663687    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token" pod "baab7f1d-fdd4-4171-9032-88660df3d519" (UID: "baab7f1d-fdd4-4171-9032-88660df3d519"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:28.764429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.764395    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhmxt\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-kube-api-access-mhmxt\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.764429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.764421    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baab7f1d-fdd4-4171-9032-88660df3d519-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.764429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.764431    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baab7f1d-fdd4-4171-9032-88660df3d519-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:28.764717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:28.764439    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baab7f1d-fdd4-4171-9032-88660df3d519-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:29.038388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.038294    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:29.066522 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066488    5915 generic.go:356] "Generic (PLEG): container finished" podID="baab7f1d-fdd4-4171-9032-88660df3d519" containerID="bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a" exitCode=0
Nov 03 21:05:29.066708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066576    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"
Nov 03 21:05:29.066708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066576    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerDied","Data":"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"}
Nov 03 21:05:29.066708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066699    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz" event={"ID":"baab7f1d-fdd4-4171-9032-88660df3d519","Type":"ContainerDied","Data":"0a4b309cd8d3d0593b9b0241addd0b1bf4cfa52a29d410a30a1c438bd95c0d0b"}
Nov 03 21:05:29.066859 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066721    5915 scope.go:122] "RemoveContainer" containerID="bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"
Nov 03 21:05:29.066859 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.066857    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"
Nov 03 21:05:29.084296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.084279    5915 scope.go:122] "RemoveContainer" containerID="ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"
Nov 03 21:05:29.094764 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.094749    5915 scope.go:122] "RemoveContainer" containerID="bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"
Nov 03 21:05:29.095090 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:29.095071    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a\": container with ID starting with bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a not found: ID does not exist" containerID="bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"
Nov 03 21:05:29.095149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.095098    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a"} err="failed to get container status \"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a\": rpc error: code = NotFound desc = could not find container \"bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a\": container with ID starting with bfde8c20dde822a7a4755965d8f5ec152099e925c3546a3d31906bf2c461828a not found: ID does not exist"
Nov 03 21:05:29.095149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.095121    5915 scope.go:122] "RemoveContainer" containerID="ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"
Nov 03 21:05:29.095413 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:29.095396    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b\": container with ID starting with ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b not found: ID does not exist" containerID="ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"
Nov 03 21:05:29.095473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.095417    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b"} err="failed to get container status \"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b\": rpc error: code = NotFound desc = could not find container \"ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b\": container with ID starting with ea3eb612f912b5f2c1ba2ce8e911657467c4cb6e22250b7097c0fa0e282a0b7b not found: ID does not exist"
Nov 03 21:05:29.237808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.237776    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:05:29.243514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.243490    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-46590df2-67b8dfd7fd-tskhq"]
Nov 03 21:05:29.355841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.355750    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:05:29.357158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.357133    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:29.360482 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.360458    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c704c5a1-openshift-default-7cbd498849-nrsdz"]
Nov 03 21:05:29.505581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.504722    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:29.689429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.689332    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2056ea49-f5a0-4021-80ed-f5b2092b8656" path="/var/lib/kubelet/pods/2056ea49-f5a0-4021-80ed-f5b2092b8656/volumes"
Nov 03 21:05:29.690105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.690066    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" path="/var/lib/kubelet/pods/baab7f1d-fdd4-4171-9032-88660df3d519/volumes"
Nov 03 21:05:29.731725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:29.731697    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:30.072581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.067982    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:30.322663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.322550    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:30.490548 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.490521    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:30.984606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.984548    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987236    5915 cpu_manager.go:414] "containerMap: removing container" podUID="2056ea49-f5a0-4021-80ed-f5b2092b8656" containerName="maas-api"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987265    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2056ea49-f5a0-4021-80ed-f5b2092b8656" containerName="maas-api"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987296    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" containerName="payload-pre-processing"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987305    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" containerName="payload-pre-processing"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987347    5915 cpu_manager.go:414] "containerMap: removing container" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987354    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987374    5915 cpu_manager.go:414] "containerMap: removing container" podUID="34527e2f-c348-41ca-b740-e6fc8d3256cc" containerName="payload-processing"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987382    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34527e2f-c348-41ca-b740-e6fc8d3256cc" containerName="payload-processing"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987411    5915 cpu_manager.go:414] "containerMap: removing container" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987418    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987477    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f9119a60-e5be-4d67-8ab4-26680eb7738b" containerName="maas-api"
Nov 03 21:05:30.987568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.987485    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9119a60-e5be-4d67-8ab4-26680eb7738b" containerName="maas-api"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988088    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="2056ea49-f5a0-4021-80ed-f5b2092b8656" containerName="maas-api"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988117    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ba05b79-4d05-441d-bcdc-6ef0b7cc980b" containerName="payload-pre-processing"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988137    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988145    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="34527e2f-c348-41ca-b740-e6fc8d3256cc" containerName="payload-processing"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988163    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="baab7f1d-fdd4-4171-9032-88660df3d519" containerName="istio-proxy"
Nov 03 21:05:30.988239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.988182    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9119a60-e5be-4d67-8ab4-26680eb7738b" containerName="maas-api"
Nov 03 21:05:30.995891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:30.995777    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.025220    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.025432    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.025473    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqmm\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.026413    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.026491    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.027282    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.027903    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.028270    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.030774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.028533    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.093957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.093905    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.302845    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.302912    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.302938    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.302971    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303114    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303140    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqmm\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303214    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303248    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303393    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.307632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.303936    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.357653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.324210    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.357653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.324499    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.370620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.367302    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:31.370620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.368046    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.370620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.368172    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.370620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.368780    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.398389 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.398352    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.398684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.358545    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-04f5216c-openshift-default-dockercfg-9gzx4\""
Nov 03 21:05:31.462912 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.461068    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.511490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.511452    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqmm\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\") pod \"e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") " pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.648695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.648586    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:31.882643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:31.869407    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:32.108886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.108850    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:05:32.142285 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:05:32.142257    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02728763_5797_4fa7_b301_1de0a913edcf.slice/crio-f2537404cd4fb151cf6e1ae64982e2df56ff45c6d79e1ff941ef473c80200bc1 WatchSource:0}: Error finding container f2537404cd4fb151cf6e1ae64982e2df56ff45c6d79e1ff941ef473c80200bc1: Status 404 returned error can't find the container with id f2537404cd4fb151cf6e1ae64982e2df56ff45c6d79e1ff941ef473c80200bc1
Nov 03 21:05:32.144921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.144892    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:05:32.144987 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.144966    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:05:32.145033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.144997    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:05:32.466855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.466818    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerStarted","Data":"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"}
Nov 03 21:05:32.467065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.466889    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerStarted","Data":"f2537404cd4fb151cf6e1ae64982e2df56ff45c6d79e1ff941ef473c80200bc1"}
Nov 03 21:05:32.590456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.590399    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podStartSLOduration=2.590385176 podStartE2EDuration="2.590385176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:05:32.585963455 +0000 UTC m=+1547.037332648" watchObservedRunningTime="2026-11-03 21:05:32.590385176 +0000 UTC m=+1547.041754352"
Nov 03 21:05:32.649220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:32.649185    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:33.397548 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:33.397513    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:33.635029 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:33.612198    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:33.815581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:33.812831    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:34.396511 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.394866    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:05:34.406796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.406123    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:05:34.406796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.406233    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.458818 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.458780    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e41159db\""
Nov 03 21:05:34.458975 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.458822    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e41159db-dockercfg-hmnkg\""
Nov 03 21:05:34.459267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.459249    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.459461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.459448    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-994b4\" (UniqueName: \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.487170 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.487131    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:34.488975 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.488954    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:05:34.560644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.560602    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.560860 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.560840    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-994b4\" (UniqueName: \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.561523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.561502    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.622383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.622352    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-994b4\" (UniqueName: \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\") pod \"payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.725011 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.724975    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:34.785838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:34.782917    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:35.012851 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.012768    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:35.167756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.167721    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:05:35.171696 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:05:35.171657    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2252259_fa76_4445_8247_4662de7f647d.slice/crio-87190a819177d59ca258d3339d3243f1387724a1fd3f892659a932b1680f8261 WatchSource:0}: Error finding container 87190a819177d59ca258d3339d3243f1387724a1fd3f892659a932b1680f8261: Status 404 returned error can't find the container with id 87190a819177d59ca258d3339d3243f1387724a1fd3f892659a932b1680f8261
Nov 03 21:05:35.296726 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.296651    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:35.481200 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.481164    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:35.560699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.560449    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" event={"ID":"f2252259-fa76-4445-8247-4662de7f647d","Type":"ContainerStarted","Data":"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"}
Nov 03 21:05:35.560699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.560515    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:35.560699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.560537    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" event={"ID":"f2252259-fa76-4445-8247-4662de7f647d","Type":"ContainerStarted","Data":"87190a819177d59ca258d3339d3243f1387724a1fd3f892659a932b1680f8261"}
Nov 03 21:05:35.687616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:35.687485    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" podStartSLOduration=1.687471843 podStartE2EDuration="1.687471843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:05:35.681241355 +0000 UTC m=+1550.132610532" watchObservedRunningTime="2026-11-03 21:05:35.687471843 +0000 UTC m=+1550.138841020"
Nov 03 21:05:37.010132 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:37.010092    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:05:37.778268 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:37.778230    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:05:37.778484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:37.778437    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy" containerID="cri-o://d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6" gracePeriod=30
Nov 03 21:05:38.559339 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:38.559309    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:38.824754 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:38.822700    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:39.104645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:39.100150    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:40.908520 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:40.908489    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:43.073751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.073724    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:05:43.129411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129381    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129429    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129633    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129694    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129684    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:43.129744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129720    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129748    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129776    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129928 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129913    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.129991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.129956    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\") pod \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\" (UID: \"53adb6c7-6789-4b49-9259-4af0abe2ad9c\") "
Nov 03 21:05:43.130050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.130026    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:43.130211 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.130181    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:43.130498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.130227    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:43.130644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.130617    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:43.131016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.130995    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.131105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.131022    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.131105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.131040    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.131105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.131056    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.131105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.131070    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.132099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.132075    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:05:43.132197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.132115    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:43.132244 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.132197    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:43.132276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.132241    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj" pod "53adb6c7-6789-4b49-9259-4af0abe2ad9c" (UID: "53adb6c7-6789-4b49-9259-4af0abe2ad9c"). InnerVolumeSpecName "kube-api-access-nwzvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:43.232460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.232427    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.232460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.232458    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.232460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.232467    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53adb6c7-6789-4b49-9259-4af0abe2ad9c-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.232779 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.232477    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwzvj\" (UniqueName: \"kubernetes.io/projected/53adb6c7-6789-4b49-9259-4af0abe2ad9c-kube-api-access-nwzvj\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:43.701204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.701175    5915 generic.go:356] "Generic (PLEG): container finished" podID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerID="d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6" exitCode=0
Nov 03 21:05:43.701326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.701233    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"
Nov 03 21:05:43.701326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.701260    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" event={"ID":"53adb6c7-6789-4b49-9259-4af0abe2ad9c","Type":"ContainerDied","Data":"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"}
Nov 03 21:05:43.701326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.701318    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb" event={"ID":"53adb6c7-6789-4b49-9259-4af0abe2ad9c","Type":"ContainerDied","Data":"b5d91f9d97db4b4ed3e4bfe5bdfbdca162ae05c08c9c61cd6da64047d95413dc"}
Nov 03 21:05:43.701447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.701340    5915 scope.go:122] "RemoveContainer" containerID="d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"
Nov 03 21:05:43.712431 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.712408    5915 scope.go:122] "RemoveContainer" containerID="d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"
Nov 03 21:05:43.712868 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:43.712848    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6\": container with ID starting with d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6 not found: ID does not exist" containerID="d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"
Nov 03 21:05:43.713027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.712876    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6"} err="failed to get container status \"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6\": rpc error: code = NotFound desc = could not find container \"d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6\": container with ID starting with d6d690b914058490626f8024965d95c52bb57f85a3b827f6e6bdabc3a28046d6 not found: ID does not exist"
Nov 03 21:05:43.868179 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.868140    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:05:43.917915 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:43.917885    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-k2hdb"]
Nov 03 21:05:45.683851 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:45.683819    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" path="/var/lib/kubelet/pods/53adb6c7-6789-4b49-9259-4af0abe2ad9c/volumes"
Nov 03 21:05:46.017656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:46.017621    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:46.570996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:46.570972    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:05:49.835961 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:49.835930    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:50.927842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:50.927810    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:55.009137 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.009102    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:05:55.009538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.009294    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" podUID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" containerName="maas-api" containerID="cri-o://ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b" gracePeriod=30
Nov 03 21:05:55.108256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.108226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:55.761184 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.761153    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:05:55.845593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.844364    5915 generic.go:356] "Generic (PLEG): container finished" podID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" containerID="ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b" exitCode=0
Nov 03 21:05:55.845593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.844537    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" event={"ID":"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef","Type":"ContainerDied","Data":"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"}
Nov 03 21:05:55.845593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.844605    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg" event={"ID":"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef","Type":"ContainerDied","Data":"51343a14b833d207b90b01874c435194bc20b060379a94ab54d9ad5fb112edb8"}
Nov 03 21:05:55.845593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.844638    5915 scope.go:122] "RemoveContainer" containerID="ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"
Nov 03 21:05:55.845593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.844840    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"
Nov 03 21:05:55.870351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.869960    5915 scope.go:122] "RemoveContainer" containerID="ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"
Nov 03 21:05:55.870691 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:05:55.870591    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b\": container with ID starting with ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b not found: ID does not exist" containerID="ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"
Nov 03 21:05:55.870691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.870632    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b"} err="failed to get container status \"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b\": rpc error: code = NotFound desc = could not find container \"ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b\": container with ID starting with ba3228aa726b7153654ee9d5b7c2526ae2c4f37f884a15659b00fe647446d23b not found: ID does not exist"
Nov 03 21:05:55.925987 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.925900    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.941954    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\") pod \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") "
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.941999    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\") pod \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") "
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.942053    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\" (UniqueName: \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\") pod \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") "
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.942209    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") pod \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\" (UID: \"3db6fb1a-b1e9-49c7-af6b-f2ad071339ef\") "
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.942336    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca" pod "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" (UID: "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:55.943606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.943130    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:55.945238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.945207    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs" pod "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" (UID: "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:55.945856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.945824    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh" pod "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" (UID: "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef"). InnerVolumeSpecName "kube-api-access-cl6hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:55.945973 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:55.945868    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls" pod "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" (UID: "3db6fb1a-b1e9-49c7-af6b-f2ad071339ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:05:56.071584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.044505    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:56.071584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.044546    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl6hh\" (UniqueName: \"kubernetes.io/projected/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-kube-api-access-cl6hh\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:56.071584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.044582    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:56.360526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.360456    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:05:56.410444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.410401    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6f2c17e9-6994f8fb89-c56hg"]
Nov 03 21:05:56.744656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:56.744626    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:57.152647 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.151247    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:57.304135 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.304106    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:57.455980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.455947    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:05:57.456186 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.456150    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" containerID="cri-o://5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf" gracePeriod=30
Nov 03 21:05:57.474374 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.474349    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:57.728819 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:57.728739    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" path="/var/lib/kubelet/pods/3db6fb1a-b1e9-49c7-af6b-f2ad071339ef/volumes"
Nov 03 21:05:58.659364 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:58.659331    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:05:58.894513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:05:58.894473    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:02.632345 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.632313    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:02.910358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.910332    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:06:02.937597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.937547    5915 generic.go:356] "Generic (PLEG): container finished" podID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerID="5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf" exitCode=0
Nov 03 21:06:02.937720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.937611    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"
Nov 03 21:06:02.937720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.937622    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerDied","Data":"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"}
Nov 03 21:06:02.937720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.937673    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" event={"ID":"d43272da-02c4-44cc-9d0b-f7f25f0cd91d","Type":"ContainerDied","Data":"28abc272c062b869397a59e0ae5aba290361166b198eb97ad7b63124049075c2"}
Nov 03 21:06:02.937720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.937699    5915 scope.go:122] "RemoveContainer" containerID="5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"
Nov 03 21:06:02.950268 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.950249    5915 scope.go:122] "RemoveContainer" containerID="edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"
Nov 03 21:06:02.960279 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.960260    5915 scope.go:122] "RemoveContainer" containerID="5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"
Nov 03 21:06:02.960575 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:02.960540    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf\": container with ID starting with 5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf not found: ID does not exist" containerID="5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"
Nov 03 21:06:02.960684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.960658    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf"} err="failed to get container status \"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf\": rpc error: code = NotFound desc = could not find container \"5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf\": container with ID starting with 5188ea1e6e6cb4c014e41e627a68a216890fb658a3cdbf6678d29422da2550cf not found: ID does not exist"
Nov 03 21:06:02.960740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.960687    5915 scope.go:122] "RemoveContainer" containerID="edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"
Nov 03 21:06:02.960961 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:02.960942    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795\": container with ID starting with edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795 not found: ID does not exist" containerID="edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"
Nov 03 21:06:02.961040 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:02.960966    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795"} err="failed to get container status \"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795\": rpc error: code = NotFound desc = could not find container \"edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795\": container with ID starting with edad77149bd7c36eb8516e2e3a8ac50cccafb7749fd0c3531a2750d88bce8795 not found: ID does not exist"
Nov 03 21:06:03.025148 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025124    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025213    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025331    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025370    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025510    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025672 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025524    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:06:03.025672 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025643    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025680    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025713    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:06:03.025788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025724    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025769    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\") pod \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\" (UID: \"d43272da-02c4-44cc-9d0b-f7f25f0cd91d\") "
Nov 03 21:06:03.025993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.025938    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:06:03.026101 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026072    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:06:03.026242 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026219    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:06:03.026863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026835    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.026972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026868    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.026972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026883    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.026972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026897    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.026972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.026911    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.027736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.027712    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:06:03.027925 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.027906    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:06:03.028122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.028103    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "kube-api-access-slmxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:06:03.028216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.028200    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy" pod "d43272da-02c4-44cc-9d0b-f7f25f0cd91d" (UID: "d43272da-02c4-44cc-9d0b-f7f25f0cd91d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:06:03.129023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.128946    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.129023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.128984    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slmxs\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-kube-api-access-slmxs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.129023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.129000    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.129023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.129013    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43272da-02c4-44cc-9d0b-f7f25f0cd91d-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:06:03.361723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.361688    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:06:03.365878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.365852    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z"]
Nov 03 21:06:03.683782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.683751    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" path="/var/lib/kubelet/pods/d43272da-02c4-44cc-9d0b-f7f25f0cd91d/volumes"
Nov 03 21:06:03.798842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.798808    5915 patch_prober.go:29] interesting pod/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:06:03.799020 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:03.798879    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-6f2c17e9-openshift-default-669594fbc7-zmg7z" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:06:05.182294 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:05.182262    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:05.365866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:05.365829    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:07.006514 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:07.006481    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:06:07.454681 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:07.454653    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:12.311402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:12.311365    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:15.985432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:15.985398    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:16.863450 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.863410    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:06:16.864163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864143    5915 cpu_manager.go:414] "containerMap: removing container" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:16.864278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864165    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:16.864278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864201    5915 cpu_manager.go:414] "containerMap: removing container" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:16.864278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864209    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:16.864278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864261    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" containerName="maas-api"
Nov 03 21:06:16.864278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864268    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" containerName="maas-api"
Nov 03 21:06:16.864544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864296    5915 cpu_manager.go:414] "containerMap: removing container" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy"
Nov 03 21:06:16.864544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864304    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy"
Nov 03 21:06:16.864544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864474    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="53adb6c7-6789-4b49-9259-4af0abe2ad9c" containerName="istio-proxy"
Nov 03 21:06:16.864544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864493    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:16.864544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.864512    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3db6fb1a-b1e9-49c7-af6b-f2ad071339ef" containerName="maas-api"
Nov 03 21:06:16.868309 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.868289    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:16.922108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.922075    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e41159db-serving-cert\""
Nov 03 21:06:16.976337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.976303    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:06:16.978774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.978742    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:16.978963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.978861    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:16.979027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.978981    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:16.979027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:16.979016    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw7vx\" (UniqueName: \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.060619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.060538    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:17.080814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.080786    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.080956 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.080854    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.080956 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.080950    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.081039 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.080981    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw7vx\" (UniqueName: \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.081098 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:17.081081    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-serving-cert: secret "maas-api-e2e-ait-e41159db-serving-cert" not found
Nov 03 21:06:17.081169 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:17.081159    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls podName:9ac4a820-85e1-42a6-8f60-b716aa29eb81 nodeName:}" failed. No retries permitted until 2026-11-03 21:06:17.581141659 +0000 UTC m=+1592.032510828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls") pod "maas-api-e2e-ait-e41159db-77d7445d46-w59kr" (UID: "9ac4a820-85e1-42a6-8f60-b716aa29eb81") : secret "maas-api-e2e-ait-e41159db-serving-cert" not found
Nov 03 21:06:17.081656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.081633    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.083627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.083608    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.164640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.161359    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw7vx\" (UniqueName: \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.266584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.265781    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:17.433941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.433821    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:17.602806 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.602774    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.605396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.605366    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") pod \"maas-api-e2e-ait-e41159db-77d7445d46-w59kr\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:17.782510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:17.782480    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:18.167092 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:06:18.167062    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac4a820_85e1_42a6_8f60_b716aa29eb81.slice/crio-1a5de9974e1c08a5c2a653a92dced8c85c7614b498c10fb196abb956fac21acf WatchSource:0}: Error finding container 1a5de9974e1c08a5c2a653a92dced8c85c7614b498c10fb196abb956fac21acf: Status 404 returned error can't find the container with id 1a5de9974e1c08a5c2a653a92dced8c85c7614b498c10fb196abb956fac21acf
Nov 03 21:06:18.167962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:18.167943    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:06:18.237517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:18.237486    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:19.090634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:19.090265    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" event={"ID":"9ac4a820-85e1-42a6-8f60-b716aa29eb81","Type":"ContainerStarted","Data":"1a5de9974e1c08a5c2a653a92dced8c85c7614b498c10fb196abb956fac21acf"}
Nov 03 21:06:20.145998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:20.145959    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" event={"ID":"9ac4a820-85e1-42a6-8f60-b716aa29eb81","Type":"ContainerStarted","Data":"e0b38ac7c86e3b9547f277c2b2948487ad27f800f382ca6ca7355a1a669a7262"}
Nov 03 21:06:20.149599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:20.149547    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:20.270706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:20.270657    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" podStartSLOduration=3.464996268 podStartE2EDuration="4.270643669s" totalImagesPullingTime="805.647401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:06:20.266138975 +0000 UTC m=+1594.717508151" watchObservedRunningTime="2026-11-03 21:06:20.270643669 +0000 UTC m=+1594.722012846"
Nov 03 21:06:22.121065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:22.121034    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:25.093293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:25.093262    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:26.209194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.209163    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:06:26.879687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.879654    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:06:26.880959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.880932    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="d43272da-02c4-44cc-9d0b-f7f25f0cd91d" containerName="istio-proxy"
Nov 03 21:06:26.885517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.885491    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:26.944117 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.944080    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:06:26.986039 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:26.986007    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-04f5216c-serving-cert\""
Nov 03 21:06:27.012119 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.012070    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.012591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.012541    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrdv\" (UniqueName: \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.012825 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.012805    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.012968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.012952    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.080167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.080135    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:27.123585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.120801    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrdv\" (UniqueName: \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.123585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.120862    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.123585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.120913    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.123585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.121185    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.163634 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:27.153740    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-serving-cert: secret "maas-api-e2e-ait-04f5216c-serving-cert" not found
Nov 03 21:06:27.163634 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:27.153828    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls podName:ca511d6f-6e55-4c78-9035-41cffa20ec65 nodeName:}" failed. No retries permitted until 2026-11-03 21:06:27.65380669 +0000 UTC m=+1602.105175867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls") pod "maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" (UID: "ca511d6f-6e55-4c78-9035-41cffa20ec65") : secret "maas-api-e2e-ait-04f5216c-serving-cert" not found
Nov 03 21:06:27.163634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.155104    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.251865 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.251829    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrdv\" (UniqueName: \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.278583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.275865    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:27.369898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.367037    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.450362 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.449331    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:27.711974 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.711870    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.717652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.717613    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") pod \"maas-api-e2e-ait-04f5216c-86f59f8758-s5thj\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:27.808598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:27.806767    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:28.391167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:28.391138    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:06:28.682207 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:28.682066    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:29.277882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.277628    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:06:29.283505 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.283475    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.394608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.393883    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-206508-openshift-default-dockercfg-stvct\""
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433238    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433328    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433358    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433442    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433479    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433509    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433675    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433734    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.441822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.433799    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnlqb\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.455780 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.453226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:29.467780 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.467747    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" event={"ID":"ca511d6f-6e55-4c78-9035-41cffa20ec65","Type":"ContainerStarted","Data":"e990391b7ad10985c2b7223c827b0fc7b42560c1f51054f5e246fd2d46f2f476"}
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536166    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536210    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536291    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536328    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536361    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.536599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536496    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.537841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.536575    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.537841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.537150    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnlqb\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.537841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.537245    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.574582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.554854    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.575914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.575881    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.576416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.576393    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.649583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.578476    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.713744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.581751    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.714041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.610326    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.898586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.890064    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:06:29.910048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.909978    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnlqb\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:29.947307 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:29.947274    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:30.036586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.035236    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\") pod \"e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") " pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:30.072627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.070209    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:30.189871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.189834    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:30.370629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.368704    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:30.657290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.642658    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:06:30.671386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.671347    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" event={"ID":"ca511d6f-6e55-4c78-9035-41cffa20ec65","Type":"ContainerStarted","Data":"790141e67bbb0581b47a750575d03b11b5bc3481d83e20a19ebc60b80f1ecf13"}
Nov 03 21:06:30.673836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.672811    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:30.703652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.703507    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:06:30.703951 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.703911    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:06:30.710905 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.710872    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:06:30.769062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.769035    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:30.795626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:30.795553    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" podStartSLOduration=3.655509955 podStartE2EDuration="4.795538587s" totalImagesPullingTime="1.140028632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:06:30.791391011 +0000 UTC m=+1605.242760187" watchObservedRunningTime="2026-11-03 21:06:30.795538587 +0000 UTC m=+1605.246907794"
Nov 03 21:06:31.682103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:31.682064    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerStarted","Data":"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"}
Nov 03 21:06:31.682496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:31.682142    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerStarted","Data":"c923418a10558056c19a262fadc18d6b235ad1bfe58ade142ec1a6339ed739a9"}
Nov 03 21:06:31.870826 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:31.870775    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podStartSLOduration=2.870755278 podStartE2EDuration="2.870755278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:06:31.870275953 +0000 UTC m=+1606.321645144" watchObservedRunningTime="2026-11-03 21:06:31.870755278 +0000 UTC m=+1606.322124454"
Nov 03 21:06:32.082589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:32.071912    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:32.082589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:32.081321    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:32.688201 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:32.688161    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:32.689306 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:32.689284    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:06:32.825603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:32.822790    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:34.730130 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:34.730095    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:34.926212 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:34.926177    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:35.083201 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:35.083126    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:37.005768 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:06:37.005733    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:06:37.694055 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:37.694030    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:06:38.063822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.063791    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:38.222209 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.222174    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:38.386640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.382941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:38.529591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.526744    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:06:38.552009 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.551972    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:38.596751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.596720    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:06:38.617601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.616439    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-206508\""
Nov 03 21:06:38.617601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.616662    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-206508-dockercfg-p98zj\""
Nov 03 21:06:38.787655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.761052    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:38.787655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.761176    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bh2\" (UniqueName: \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:38.961846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.949851    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:38.961846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.949996    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bh2\" (UniqueName: \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:38.961846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:38.952270    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:39.025048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.025013    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bh2\" (UniqueName: \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\") pod \"payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:39.029159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.029136    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:06:39.039691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.037396    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.042726 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.042625    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:06:39.052785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.052477    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlhvm\" (UniqueName: \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.053074 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.053046    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.155138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.155101    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhvm\" (UniqueName: \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.155508 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.155153    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.156142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.156114    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.216017 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.215986    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlhvm\" (UniqueName: \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\") pod \"payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.233980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.233950    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:39.370765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.370737    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.616426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.616401    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:06:39.617466 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:06:39.617441    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7b2612_576e_43ed_b659_13220ac034c1.slice/crio-0c02c6a63959ad836875b38306095c449e506511f1bd88fe216ffae52ed0815c WatchSource:0}: Error finding container 0c02c6a63959ad836875b38306095c449e506511f1bd88fe216ffae52ed0815c: Status 404 returned error can't find the container with id 0c02c6a63959ad836875b38306095c449e506511f1bd88fe216ffae52ed0815c
Nov 03 21:06:39.762156 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:06:39.762118    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7c2529_c044_4830_a99e_ebb22e77acb3.slice/crio-e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba WatchSource:0}: Error finding container e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba: Status 404 returned error can't find the container with id e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba
Nov 03 21:06:39.762357 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.762336    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:06:39.974131 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.974099    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" event={"ID":"5d7c2529-c044-4830-a99e-ebb22e77acb3","Type":"ContainerStarted","Data":"5e52fb4b50ec9139e314666a3c472a635f37f704fd5f31fe418cfd2307f2759a"}
Nov 03 21:06:39.974329 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.974155    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" event={"ID":"5d7c2529-c044-4830-a99e-ebb22e77acb3","Type":"ContainerStarted","Data":"e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba"}
Nov 03 21:06:39.974329 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.974185    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:39.976041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.976010    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" event={"ID":"5f7b2612-576e-43ed-b659-13220ac034c1","Type":"ContainerStarted","Data":"25b76d38db47aebffbe698c05d98f176d1737d8fa20d9b162fa37a15674f5a40"}
Nov 03 21:06:39.976164 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.976056    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" event={"ID":"5f7b2612-576e-43ed-b659-13220ac034c1","Type":"ContainerStarted","Data":"0c02c6a63959ad836875b38306095c449e506511f1bd88fe216ffae52ed0815c"}
Nov 03 21:06:39.976164 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:39.976105    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:40.100972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:40.100887    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" podStartSLOduration=2.100871152 podStartE2EDuration="2.100871152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:06:40.094811817 +0000 UTC m=+1614.546180999" watchObservedRunningTime="2026-11-03 21:06:40.100871152 +0000 UTC m=+1614.552240328"
Nov 03 21:06:40.260695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:40.260134    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" podStartSLOduration=2.260112151 podStartE2EDuration="2.260112151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:06:40.234142534 +0000 UTC m=+1614.685511711" watchObservedRunningTime="2026-11-03 21:06:40.260112151 +0000 UTC m=+1614.711481329"
Nov 03 21:06:43.208849 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:43.208798    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:48.007593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:48.007523    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:50.443480 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:50.443447    5915 scope.go:122] "RemoveContainer" containerID="ae367318b4c2b36619346ab27a07211eeb3fd077ecca247d33e60db58124364d"
Nov 03 21:06:50.988504 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:50.988474    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:06:50.988718 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:50.988541    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:06:53.016769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:53.016711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:06:58.835543 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:06:58.835447    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:01.344712 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:01.344663    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:07:01.749584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:01.749326    5915 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/health\": context deadline exceeded"
Nov 03 21:07:02.830694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:02.830656    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:03.104642 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:03.102112    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:05.061490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:05.061448    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:07.008652 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:07.008615    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:07:08.093996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:08.093967    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:09.529802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.529724    5915 generic.go:356] "Generic (PLEG): container finished" podID="02728763-5797-4fa7-b301-1de0a913edcf" containerID="5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841" exitCode=0
Nov 03 21:07:09.530228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.529796    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerDied","Data":"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"}
Nov 03 21:07:09.530276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.530260    5915 scope.go:122] "RemoveContainer" containerID="5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"
Nov 03 21:07:09.531908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.531879    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:07:09.531995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.531944    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:07:09.531995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:09.531976    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:07:10.375421 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:10.375386    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:10.556899 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:10.556858    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerStarted","Data":"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"}
Nov 03 21:07:10.648998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:10.648928    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:07:10.649153 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:10.649111    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:10.649197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:10.649165    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:11.649777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:11.649743    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:11.650294 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:11.649805    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:12.051889 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:12.051858    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"]
Nov 03 21:07:12.124999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:12.124964    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-66836146-jhtkq"]
Nov 03 21:07:12.649033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:12.649002    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:12.649205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:12.649050    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:13.649228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:13.649190    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:13.649685 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:13.649246    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:13.682931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:13.682901    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e478dc7-f5be-4029-be72-d44a7cef8211" path="/var/lib/kubelet/pods/4e478dc7-f5be-4029-be72-d44a7cef8211/volumes"
Nov 03 21:07:14.649757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:14.649710    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:14.650280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:14.649773    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:14.676062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:14.676030    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:15.530290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:15.530246    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:15.677319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:15.677264    5915 patch_prober.go:29] interesting pod/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 03 21:07:15.678006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:15.677972    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 03 21:07:16.308149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:16.308107    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:16.655730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:16.655636    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:07:16.656155 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:16.656132    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:07:16.657006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:16.656963    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:07:20.059585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:20.059536    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:20.123591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:20.122230    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"]
Nov 03 21:07:20.126722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:20.126697    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5f0ac546-w4d2l"]
Nov 03 21:07:20.231968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:20.231939    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:20.781289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:20.781250    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:21.051933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:21.051859    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:21.683071 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:21.683033    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f9fc69d-3f1b-4576-beec-ad4b70282504" path="/var/lib/kubelet/pods/3f9fc69d-3f1b-4576-beec-ad4b70282504/volumes"
Nov 03 21:07:24.653998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:24.653963    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:07:24.654416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:24.654339    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy" containerID="cri-o://bb1d9b7488d64d4508a0d57c3d28bce272cacb8ead860ea4c5942186f83760ba" gracePeriod=30
Nov 03 21:07:25.908285 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:25.908251    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:29.883515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:29.883480    5915 generic.go:356] "Generic (PLEG): container finished" podID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerID="bb1d9b7488d64d4508a0d57c3d28bce272cacb8ead860ea4c5942186f83760ba" exitCode=0
Nov 03 21:07:29.884025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:29.883539    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" event={"ID":"4e678505-5cc0-4f23-9e3c-2dfbcca7a334","Type":"ContainerDied","Data":"bb1d9b7488d64d4508a0d57c3d28bce272cacb8ead860ea4c5942186f83760ba"}
Nov 03 21:07:30.051348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.051314    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117406    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117474    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117635    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117713    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117754    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117808    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117840    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.117898    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.118036    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\") pod \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\" (UID: \"4e678505-5cc0-4f23-9e3c-2dfbcca7a334\") "
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.125972    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:30.132578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.126292    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:30.133663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.133393    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:30.134233 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.133913    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:30.164517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.164435    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "kube-api-access-tbprz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:30.165090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.165052    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:30.169355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.169310    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:30.179918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.179688    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:30.180137 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.180110    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy" pod "4e678505-5cc0-4f23-9e3c-2dfbcca7a334" (UID: "4e678505-5cc0-4f23-9e3c-2dfbcca7a334"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:30.225732 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.225389    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227413 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227389    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbprz\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-kube-api-access-tbprz\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227611    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227724    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227816    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227910 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227899    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.227990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.227979    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.228063 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.228053    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.228141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.228130    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e678505-5cc0-4f23-9e3c-2dfbcca7a334-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:30.654166 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.654134    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:30.894911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.894885    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"
Nov 03 21:07:30.895373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.894916    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd" event={"ID":"4e678505-5cc0-4f23-9e3c-2dfbcca7a334","Type":"ContainerDied","Data":"0896f2d0ed2d04a3a986cb44868c7e44eeacf4f89eca8572c5cbc2d747eb3b2a"}
Nov 03 21:07:30.895373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.894990    5915 scope.go:122] "RemoveContainer" containerID="bb1d9b7488d64d4508a0d57c3d28bce272cacb8ead860ea4c5942186f83760ba"
Nov 03 21:07:30.958188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:30.958157    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:31.131497 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:31.131464    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:07:31.139061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:31.139024    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-wc6nd"]
Nov 03 21:07:31.683062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:31.683031    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" path="/var/lib/kubelet/pods/4e678505-5cc0-4f23-9e3c-2dfbcca7a334/volumes"
Nov 03 21:07:35.828416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.828386    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:07:35.829133 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.829117    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy"
Nov 03 21:07:35.829180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.829134    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy"
Nov 03 21:07:35.829264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.829255    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e678505-5cc0-4f23-9e3c-2dfbcca7a334" containerName="istio-proxy"
Nov 03 21:07:35.832885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.832867    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:35.885392 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.885368    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-206508-serving-cert\""
Nov 03 21:07:35.914043 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.914013    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:35.914183 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.914058    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:35.914396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.914374    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb6kf\" (UniqueName: \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:35.914437 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.914409    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:35.940803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:35.940773    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:07:36.015325 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.015290    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb6kf\" (UniqueName: \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.015509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.015334    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.015607 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.015523    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.015992 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.015709    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.015992 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:36.015907    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-serving-cert: secret "maas-api-e2e-shared-a-w1-206508-serving-cert" not found
Nov 03 21:07:36.016195 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:36.016002    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls podName:0c0c6ad9-eabd-4b30-9424-1e841aba47ff nodeName:}" failed. No retries permitted until 2026-11-03 21:07:36.515979371 +0000 UTC m=+1670.967348527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls") pod "maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" (UID: "0c0c6ad9-eabd-4b30-9424-1e841aba47ff") : secret "maas-api-e2e-shared-a-w1-206508-serving-cert" not found
Nov 03 21:07:36.016340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.016321    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.018440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.018419    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.034595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.034510    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:36.143794 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.143675    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb6kf\" (UniqueName: \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.227390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.227312    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:36.388232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.382735    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:36.572876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.572847    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.575402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.575381    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:36.746738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:36.746710    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:37.005901 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:37.005866    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:07:37.130754 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:37.130724    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:07:37.131122 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:07:37.131095    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0c6ad9_eabd_4b30_9424_1e841aba47ff.slice/crio-fcbece92cc0ab87c34b2f67240f452a011ec591bf4f5f839aa8a1cba2acf8b46 WatchSource:0}: Error finding container fcbece92cc0ab87c34b2f67240f452a011ec591bf4f5f839aa8a1cba2acf8b46: Status 404 returned error can't find the container with id fcbece92cc0ab87c34b2f67240f452a011ec591bf4f5f839aa8a1cba2acf8b46
Nov 03 21:07:37.133016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:37.132997    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:07:37.958225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:37.958189    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" event={"ID":"0c0c6ad9-eabd-4b30-9424-1e841aba47ff","Type":"ContainerStarted","Data":"fcbece92cc0ab87c34b2f67240f452a011ec591bf4f5f839aa8a1cba2acf8b46"}
Nov 03 21:07:38.919768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.919735    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:07:38.920265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.919987    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy" containerID="cri-o://69632878bcbd13bff2b3a4eae144f7e8df4a7a2e4d766ae61a2ec788d43d83a1" gracePeriod=30
Nov 03 21:07:38.970256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.970176    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" event={"ID":"0c0c6ad9-eabd-4b30-9424-1e841aba47ff","Type":"ContainerStarted","Data":"454b08b42d40c620095011e4b977948ce508782aca2f7ba402aeccf9a11f59f9"}
Nov 03 21:07:38.970694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.970671    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:38.971062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.971049    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:07:38.971254 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.971233    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy" containerID="cri-o://b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09" gracePeriod=30
Nov 03 21:07:38.977399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.977370    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 21:07:38.977617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:38.977595    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy" containerID="cri-o://3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86" gracePeriod=30
Nov 03 21:07:39.245581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:39.242214    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" podStartSLOduration=3.344845388 podStartE2EDuration="4.242191761s" totalImagesPullingTime="897.346373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:07:39.229617733 +0000 UTC m=+1673.680986914" watchObservedRunningTime="2026-11-03 21:07:39.242191761 +0000 UTC m=+1673.693560938"
Nov 03 21:07:39.739004 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:39.738962    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:07:39.739217 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:39.739192    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy" containerID="cri-o://0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01" gracePeriod=30
Nov 03 21:07:41.438612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:41.438579    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:43.701268 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:43.700175    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:07:43.947435 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:43.947375    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:44.289383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.289198    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:44.308606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.308208    5915 generic.go:356] "Generic (PLEG): container finished" podID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerID="69632878bcbd13bff2b3a4eae144f7e8df4a7a2e4d766ae61a2ec788d43d83a1" exitCode=0
Nov 03 21:07:44.308606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.308446    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" event={"ID":"37882b8d-ba0a-41b3-b1d6-f8c896248387","Type":"ContainerDied","Data":"69632878bcbd13bff2b3a4eae144f7e8df4a7a2e4d766ae61a2ec788d43d83a1"}
Nov 03 21:07:44.517960 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.517930    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:07:44.531652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.531625    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 21:07:44.561777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.561740    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:07:44.607443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607415    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607453    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607482    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607503    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607527    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607613    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.607655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607636    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607662    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607726    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607752    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607777    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607773    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607810    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607841    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607868    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607921    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607959    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.607987    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608027    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608069    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608083    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608139    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608169    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608220    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608254    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608296    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608338    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608395    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608437    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\") pod \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\" (UID: \"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608490    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608624    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\") pod \"37882b8d-ba0a-41b3-b1d6-f8c896248387\" (UID: \"37882b8d-ba0a-41b3-b1d6-f8c896248387\") "
Nov 03 21:07:44.608937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608672    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\") pod \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\" (UID: \"7175c94c-c24a-4912-a86f-f0c9b97ac5ec\") "
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.608975    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609174    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609400    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609484    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609504    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609518    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609574    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.609585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609589    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.610054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609603    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.610054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609616    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.610054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.609871    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:44.610230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.610159    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:44.610419 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.610365    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.610691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.610658    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.610691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.610677    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.610870 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.610727    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.611089 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.611031    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.611409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.611380    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "kube-api-access-d6v9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.612001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.611977    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.612962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.612858    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.612962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.612921    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:44.612962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.612927    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "kube-api-access-vv8mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.612962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.612959    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.613164 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.612973    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.613641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.613618    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.613938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.613912    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy" pod "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" (UID: "368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:44.614289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.614248    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:44.614361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.614288    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo" pod "37882b8d-ba0a-41b3-b1d6-f8c896248387" (UID: "37882b8d-ba0a-41b3-b1d6-f8c896248387"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:44.614495 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.614475    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr" pod "7175c94c-c24a-4912-a86f-f0c9b97ac5ec" (UID: "7175c94c-c24a-4912-a86f-f0c9b97ac5ec"). InnerVolumeSpecName "kube-api-access-856dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:44.711698 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711662    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6v9l\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-kube-api-access-d6v9l\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.711698 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711698    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711714    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711729    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711744    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711758    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37882b8d-ba0a-41b3-b1d6-f8c896248387-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711771    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711784    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711801    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711816    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711832    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv8mm\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-kube-api-access-vv8mm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711847    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711863    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711879    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711897    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711920    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711932    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-856dr\" (UniqueName: \"kubernetes.io/projected/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-kube-api-access-856dr\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711942    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7175c94c-c24a-4912-a86f-f0c9b97ac5ec-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711952    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.712109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.711961    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37882b8d-ba0a-41b3-b1d6-f8c896248387-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:44.985397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:44.985370    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:07:45.060678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.059452    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:07:45.245773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.244217    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.245773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.244309    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.245773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.244348    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.245773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.244454    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.255251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.255201    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.255756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.255727    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.255880 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.255861    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.255971 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.255955    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.256059 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.256045    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.256134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.256121    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.256313 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.256299    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\") pod \"649ea92f-4fd0-4e09-8299-8977c6383d1a\" (UID: \"649ea92f-4fd0-4e09-8299-8977c6383d1a\") "
Nov 03 21:07:45.260836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.257097    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.278240 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.278170    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.295639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.282837    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.304599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.303825    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.376723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.372053    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:45.376723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.374099    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:45.382630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.378534    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.393334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.380011    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "kube-api-access-qvsrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.393703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.393234    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/649ea92f-4fd0-4e09-8299-8977c6383d1a-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.393703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.393635    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.393703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.393654    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.393703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.393670    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.394001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.393111    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy" pod "649ea92f-4fd0-4e09-8299-8977c6383d1a" (UID: "649ea92f-4fd0-4e09-8299-8977c6383d1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.405580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.394415    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"
Nov 03 21:07:45.413083 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.413026    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk" event={"ID":"37882b8d-ba0a-41b3-b1d6-f8c896248387","Type":"ContainerDied","Data":"4073b58a5e84df4c67002ec16e9769d3c50626d2c2e399e22f61d283c11218e1"}
Nov 03 21:07:45.413382 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.413364    5915 scope.go:122] "RemoveContainer" containerID="69632878bcbd13bff2b3a4eae144f7e8df4a7a2e4d766ae61a2ec788d43d83a1"
Nov 03 21:07:45.417247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.417215    5915 generic.go:356] "Generic (PLEG): container finished" podID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerID="3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86" exitCode=0
Nov 03 21:07:45.417541 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.417510    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerDied","Data":"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"}
Nov 03 21:07:45.417541 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.417577    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"
Nov 03 21:07:45.417541 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.417610    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h" event={"ID":"368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b","Type":"ContainerDied","Data":"f01ece02987ea7cb71d74b1a14fde76f92403eb3dec532e43f25d898feae3506"}
Nov 03 21:07:45.419658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.419636    5915 generic.go:356] "Generic (PLEG): container finished" podID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerID="b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09" exitCode=0
Nov 03 21:07:45.419796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.419783    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" event={"ID":"7175c94c-c24a-4912-a86f-f0c9b97ac5ec","Type":"ContainerDied","Data":"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"}
Nov 03 21:07:45.419855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.419822    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b" event={"ID":"7175c94c-c24a-4912-a86f-f0c9b97ac5ec","Type":"ContainerDied","Data":"b43e85209a28e6b4215bcbaaa0afd851508b05bd688d73e4d2a8877444d55258"}
Nov 03 21:07:45.419910 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.419903    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"
Nov 03 21:07:45.422216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.422187    5915 generic.go:356] "Generic (PLEG): container finished" podID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerID="0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01" exitCode=0
Nov 03 21:07:45.422411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.422382    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" event={"ID":"649ea92f-4fd0-4e09-8299-8977c6383d1a","Type":"ContainerDied","Data":"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"}
Nov 03 21:07:45.422606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.422589    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn" event={"ID":"649ea92f-4fd0-4e09-8299-8977c6383d1a","Type":"ContainerDied","Data":"21b65ca07ae9a3473d0e5c2a861d89970e28cded7aabaa3d210c608944d49fbd"}
Nov 03 21:07:45.422908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.422893    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"
Nov 03 21:07:45.430059 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.430035    5915 scope.go:122] "RemoveContainer" containerID="3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"
Nov 03 21:07:45.449524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.449505    5915 scope.go:122] "RemoveContainer" containerID="70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"
Nov 03 21:07:45.464174 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.464151    5915 scope.go:122] "RemoveContainer" containerID="3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"
Nov 03 21:07:45.464476 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:45.464454    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86\": container with ID starting with 3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86 not found: ID does not exist" containerID="3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"
Nov 03 21:07:45.464601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.464490    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86"} err="failed to get container status \"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86\": rpc error: code = NotFound desc = could not find container \"3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86\": container with ID starting with 3847e54199d74f8b01b936d2ac9e88897ffee60b246e5d3966b563e0d8cebe86 not found: ID does not exist"
Nov 03 21:07:45.464601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.464516    5915 scope.go:122] "RemoveContainer" containerID="70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"
Nov 03 21:07:45.464867 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:45.464850    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5\": container with ID starting with 70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5 not found: ID does not exist" containerID="70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"
Nov 03 21:07:45.464908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.464876    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5"} err="failed to get container status \"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5\": rpc error: code = NotFound desc = could not find container \"70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5\": container with ID starting with 70e924f7b3feb8e6cf33e0a0e5192f2425dbb4beafd2a3400a52455d11da7fd5 not found: ID does not exist"
Nov 03 21:07:45.464953 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.464906    5915 scope.go:122] "RemoveContainer" containerID="b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"
Nov 03 21:07:45.475087 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.475065    5915 scope.go:122] "RemoveContainer" containerID="b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"
Nov 03 21:07:45.475363 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:45.475339    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09\": container with ID starting with b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09 not found: ID does not exist" containerID="b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"
Nov 03 21:07:45.475442 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.475375    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09"} err="failed to get container status \"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09\": rpc error: code = NotFound desc = could not find container \"b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09\": container with ID starting with b736a228cfb69fae3c86cd3c1b0e654380a10d6205bdc918a8765062b9b13e09 not found: ID does not exist"
Nov 03 21:07:45.475442 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.475398    5915 scope.go:122] "RemoveContainer" containerID="0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"
Nov 03 21:07:45.486232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.486217    5915 scope.go:122] "RemoveContainer" containerID="0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"
Nov 03 21:07:45.486782 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:45.486701    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01\": container with ID starting with 0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01 not found: ID does not exist" containerID="0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"
Nov 03 21:07:45.486782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.486736    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01"} err="failed to get container status \"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01\": rpc error: code = NotFound desc = could not find container \"0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01\": container with ID starting with 0eb7531796391768fb4a2e598814c066a9164f21d6676871445f9a5052618a01 not found: ID does not exist"
Nov 03 21:07:45.594637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.591869    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvsrw\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-kube-api-access-qvsrw\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.594637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.591910    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.594637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.591926    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/649ea92f-4fd0-4e09-8299-8977c6383d1a-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.674592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.667647    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:07:45.726105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.726076    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-lzrbk"]
Nov 03 21:07:45.756645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.756618    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:07:45.760503 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.760480    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lqhmn"]
Nov 03 21:07:45.871109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.871039    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 21:07:45.877149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.877125    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-h8n9h"]
Nov 03 21:07:45.986776 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.986743    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:07:45.996519 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:45.996487    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-kb25b"]
Nov 03 21:07:46.508914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:46.508883    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:46.694242 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:46.692367    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:46.842178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:46.842080    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:47.533548 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:47.533516    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:47.794419 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:47.794327    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" path="/var/lib/kubelet/pods/368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b/volumes"
Nov 03 21:07:47.795433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:47.795410    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" path="/var/lib/kubelet/pods/37882b8d-ba0a-41b3-b1d6-f8c896248387/volumes"
Nov 03 21:07:47.796247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:47.796227    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" path="/var/lib/kubelet/pods/649ea92f-4fd0-4e09-8299-8977c6383d1a/volumes"
Nov 03 21:07:47.797406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:47.797388    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" path="/var/lib/kubelet/pods/7175c94c-c24a-4912-a86f-f0c9b97ac5ec/volumes"
Nov 03 21:07:48.701634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:48.701595    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" podUID="f2252259-fa76-4445-8247-4662de7f647d" containerName="payload-pre-processing" containerID="cri-o://f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019" gracePeriod=25
Nov 03 21:07:48.995843 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:48.995809    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:07:49.011095 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.010992    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\" (UniqueName: \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\") pod \"f2252259-fa76-4445-8247-4662de7f647d\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") "
Nov 03 21:07:49.011247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.011162    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\") pod \"f2252259-fa76-4445-8247-4662de7f647d\" (UID: \"f2252259-fa76-4445-8247-4662de7f647d\") "
Nov 03 21:07:49.011644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.011613    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume" pod "f2252259-fa76-4445-8247-4662de7f647d" (UID: "f2252259-fa76-4445-8247-4662de7f647d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:49.012061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.012038    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2252259-fa76-4445-8247-4662de7f647d-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:49.013264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.013240    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4" pod "f2252259-fa76-4445-8247-4662de7f647d" (UID: "f2252259-fa76-4445-8247-4662de7f647d"). InnerVolumeSpecName "kube-api-access-994b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:49.113661 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.113627    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-994b4\" (UniqueName: \"kubernetes.io/projected/f2252259-fa76-4445-8247-4662de7f647d-kube-api-access-994b4\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:49.460190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.460155    5915 generic.go:356] "Generic (PLEG): container finished" podID="f2252259-fa76-4445-8247-4662de7f647d" containerID="f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019" exitCode=0
Nov 03 21:07:49.460394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.460200    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"
Nov 03 21:07:49.460394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.460245    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" event={"ID":"f2252259-fa76-4445-8247-4662de7f647d","Type":"ContainerDied","Data":"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"}
Nov 03 21:07:49.460394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.460316    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q" event={"ID":"f2252259-fa76-4445-8247-4662de7f647d","Type":"ContainerDied","Data":"87190a819177d59ca258d3339d3243f1387724a1fd3f892659a932b1680f8261"}
Nov 03 21:07:49.460394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.460339    5915 scope.go:122] "RemoveContainer" containerID="f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"
Nov 03 21:07:49.471725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.471704    5915 scope.go:122] "RemoveContainer" containerID="f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"
Nov 03 21:07:49.472048 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:07:49.472027    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019\": container with ID starting with f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019 not found: ID does not exist" containerID="f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"
Nov 03 21:07:49.472163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.472058    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019"} err="failed to get container status \"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019\": rpc error: code = NotFound desc = could not find container \"f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019\": container with ID starting with f416630977ef2f1bac996ac18de61bff8d850a979836aa9e2dc79530284e3019 not found: ID does not exist"
Nov 03 21:07:49.627069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.627033    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:07:49.682770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:49.682739    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e41159db-bb465588c-4g75q"]
Nov 03 21:07:51.630052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:51.630021    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:51.738169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:51.738135    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2252259-fa76-4445-8247-4662de7f647d" path="/var/lib/kubelet/pods/f2252259-fa76-4445-8247-4662de7f647d/volumes"
Nov 03 21:07:54.062513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:54.062471    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:54.213424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:54.213379    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:56.436646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:56.436610    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:07:59.808359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:07:59.808322    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:00.660066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:00.660036    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:08:00.660342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:00.660291    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" podUID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" containerName="maas-api" containerID="cri-o://c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e" gracePeriod=30
Nov 03 21:08:00.766856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:00.766820    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:01.448641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.448488    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:08:01.551697 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.551628    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") pod \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") "
Nov 03 21:08:01.551697 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.551683    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\" (UniqueName: \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\") pod \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") "
Nov 03 21:08:01.551917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.551767    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\") pod \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") "
Nov 03 21:08:01.551917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.551909    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\") pod \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\" (UID: \"5318bf4b-8fc2-42c9-b540-35e2ac66ce98\") "
Nov 03 21:08:01.552314 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.552284    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca" pod "5318bf4b-8fc2-42c9-b540-35e2ac66ce98" (UID: "5318bf4b-8fc2-42c9-b540-35e2ac66ce98"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:01.552658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.552630    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:01.554160 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.554137    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs" pod "5318bf4b-8fc2-42c9-b540-35e2ac66ce98" (UID: "5318bf4b-8fc2-42c9-b540-35e2ac66ce98"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:01.554258 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.554184    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls" pod "5318bf4b-8fc2-42c9-b540-35e2ac66ce98" (UID: "5318bf4b-8fc2-42c9-b540-35e2ac66ce98"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:01.554258 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.554228    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg" pod "5318bf4b-8fc2-42c9-b540-35e2ac66ce98" (UID: "5318bf4b-8fc2-42c9-b540-35e2ac66ce98"). InnerVolumeSpecName "kube-api-access-67jrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:01.582939 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.582907    5915 generic.go:356] "Generic (PLEG): container finished" podID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" containerID="c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e" exitCode=0
Nov 03 21:08:01.583038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.582967    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"
Nov 03 21:08:01.583038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.582990    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" event={"ID":"5318bf4b-8fc2-42c9-b540-35e2ac66ce98","Type":"ContainerDied","Data":"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"}
Nov 03 21:08:01.583118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.583065    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt" event={"ID":"5318bf4b-8fc2-42c9-b540-35e2ac66ce98","Type":"ContainerDied","Data":"c81557af473667be5a2f3119088cce588e2c883f6e44cc2f91bc4c00f1cc1412"}
Nov 03 21:08:01.583118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.583087    5915 scope.go:122] "RemoveContainer" containerID="c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"
Nov 03 21:08:01.596842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.596817    5915 scope.go:122] "RemoveContainer" containerID="c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"
Nov 03 21:08:01.597192 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:01.597168    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e\": container with ID starting with c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e not found: ID does not exist" containerID="c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"
Nov 03 21:08:01.597293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.597200    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e"} err="failed to get container status \"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e\": rpc error: code = NotFound desc = could not find container \"c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e\": container with ID starting with c19c9a0324d8c208b4aa3b5593a7365961458f0e8463aadd25de09715f801e1e not found: ID does not exist"
Nov 03 21:08:01.653802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.653770    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:01.653802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.653798    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:01.654013 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.653814    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67jrg\" (UniqueName: \"kubernetes.io/projected/5318bf4b-8fc2-42c9-b540-35e2ac66ce98-kube-api-access-67jrg\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:01.757411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.757383    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:08:01.761080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:01.761055    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e3917193-76bcb77db8-4dhlt"]
Nov 03 21:08:02.597169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:02.597137    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:02.829407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:02.829377    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:02.994578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:02.993521    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:03.572185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:03.572152    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:03.682957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:03.682924    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" path="/var/lib/kubelet/pods/5318bf4b-8fc2-42c9-b540-35e2ac66ce98/volumes"
Nov 03 21:08:06.157917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:06.157886    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:07.005133 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:07.005096    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:08:08.049743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:08.049709    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:09.614075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:09.614047    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:09.790945 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:09.790908    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:10.044426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.044391    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:10.375863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.375834    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 21:08:10.376061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.376035    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" containerID="cri-o://bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00" gracePeriod=30
Nov 03 21:08:10.426656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.426629    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 21:08:10.426872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.426846    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" containerID="cri-o://70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a" gracePeriod=30
Nov 03 21:08:10.432042 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.432009    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 21:08:10.432248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.432224    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" containerID="cri-o://affd672a9047aa7159582f2bce5793afba807b2efd78fd36ac44a4cabdf871e2" gracePeriod=30
Nov 03 21:08:10.475469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.475428    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 21:08:10.475702 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.475667    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" containerID="cri-o://6148435255fffcc9ebc2972b6f5d6f70e1db57d0250da428ec084bcdba422ef0" gracePeriod=30
Nov 03 21:08:10.538510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.538472    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:10.702837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.702811    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:10.770009 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.769978    5915 generic.go:356] "Generic (PLEG): container finished" podID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerID="cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b" exitCode=0
Nov 03 21:08:10.770177 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.770051    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerDied","Data":"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"}
Nov 03 21:08:10.770523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.770508    5915 scope.go:122] "RemoveContainer" containerID="cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"
Nov 03 21:08:10.772102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.772075    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:10.772187 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.772143    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:10.772187 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.772177    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:10.987979 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:10.987893    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:11.780052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:11.780016    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerStarted","Data":"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"}
Nov 03 21:08:12.071690 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.071587    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:08:12.071867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.071814    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:12.071940 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.071870    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:12.524250 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.524201    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:08:12.524506 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.524466    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" podUID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" containerName="maas-api" containerID="cri-o://e0b38ac7c86e3b9547f277c2b2948487ad27f800f382ca6ca7355a1a669a7262" gracePeriod=30
Nov 03 21:08:12.629304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.629268    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:12.805258 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.805023    5915 generic.go:356] "Generic (PLEG): container finished" podID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" containerID="e0b38ac7c86e3b9547f277c2b2948487ad27f800f382ca6ca7355a1a669a7262" exitCode=0
Nov 03 21:08:12.806878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:12.806849    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" event={"ID":"9ac4a820-85e1-42a6-8f60-b716aa29eb81","Type":"ContainerDied","Data":"e0b38ac7c86e3b9547f277c2b2948487ad27f800f382ca6ca7355a1a669a7262"}
Nov 03 21:08:13.081276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.081186    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:13.081533 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.081502    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:13.175190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.175153    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:08:13.324207 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.324164    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\" (UniqueName: \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\") pod \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") "
Nov 03 21:08:13.324418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.324254    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\") pod \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") "
Nov 03 21:08:13.324418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.324304    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") pod \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") "
Nov 03 21:08:13.324418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.324401    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\") pod \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\" (UID: \"9ac4a820-85e1-42a6-8f60-b716aa29eb81\") "
Nov 03 21:08:13.325117 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.324997    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca" pod "9ac4a820-85e1-42a6-8f60-b716aa29eb81" (UID: "9ac4a820-85e1-42a6-8f60-b716aa29eb81"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:13.326949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.326921    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls" pod "9ac4a820-85e1-42a6-8f60-b716aa29eb81" (UID: "9ac4a820-85e1-42a6-8f60-b716aa29eb81"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:13.327455 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.327436    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx" pod "9ac4a820-85e1-42a6-8f60-b716aa29eb81" (UID: "9ac4a820-85e1-42a6-8f60-b716aa29eb81"). InnerVolumeSpecName "kube-api-access-kw7vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:13.327987 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.327962    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs" pod "9ac4a820-85e1-42a6-8f60-b716aa29eb81" (UID: "9ac4a820-85e1-42a6-8f60-b716aa29eb81"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:13.372346 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.372311    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:08:13.372461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.372366    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:08:13.428585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.428241    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw7vx\" (UniqueName: \"kubernetes.io/projected/9ac4a820-85e1-42a6-8f60-b716aa29eb81-kube-api-access-kw7vx\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:13.428585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.428279    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ac4a820-85e1-42a6-8f60-b716aa29eb81-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:13.428585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.428295    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:13.428585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.428310    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ac4a820-85e1-42a6-8f60-b716aa29eb81-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:13.862608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.856734    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr" event={"ID":"9ac4a820-85e1-42a6-8f60-b716aa29eb81","Type":"ContainerDied","Data":"1a5de9974e1c08a5c2a653a92dced8c85c7614b498c10fb196abb956fac21acf"}
Nov 03 21:08:13.862608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.856802    5915 scope.go:122] "RemoveContainer" containerID="e0b38ac7c86e3b9547f277c2b2948487ad27f800f382ca6ca7355a1a669a7262"
Nov 03 21:08:13.862608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:13.856963    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"
Nov 03 21:08:14.067634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.063094    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:08:14.074090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.074058    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:14.074381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.074353    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:14.112734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.110505    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e41159db-77d7445d46-w59kr"]
Nov 03 21:08:14.461359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.461324    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:08:14.461521 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.461388    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:08:14.596284 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.596251    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:14.799916 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.799711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:14.992591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:14.988470    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:15.071599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.071486    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:15.071599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.071551    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:15.712929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.710357    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" path="/var/lib/kubelet/pods/9ac4a820-85e1-42a6-8f60-b716aa29eb81/volumes"
Nov 03 21:08:15.846836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.846809    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 21:08:15.873014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.872902    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 21:08:15.874825 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.874800    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.874945 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.874929    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875035    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875090    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.875144 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875122    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875180    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875224    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875274 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875256    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875274 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875253    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.875339 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875318    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875407    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\") pod \"d47c5e43-c802-4ba3-a503-b33787be0662\" (UID: \"d47c5e43-c802-4ba3-a503-b33787be0662\") "
Nov 03 21:08:15.875493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875477    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875683    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.875887    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.876187    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.876211    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47c5e43-c802-4ba3-a503-b33787be0662-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.876225    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.876241    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.876659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.876257    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.878030 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.878003    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "kube-api-access-7nqmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:15.878611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.878341    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:15.879089 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.879046    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:08:15.881490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.880279    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy" pod "d47c5e43-c802-4ba3-a503-b33787be0662" (UID: "d47c5e43-c802-4ba3-a503-b33787be0662"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.881490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.881178    5915 generic.go:356] "Generic (PLEG): container finished" podID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerID="affd672a9047aa7159582f2bce5793afba807b2efd78fd36ac44a4cabdf871e2" exitCode=0
Nov 03 21:08:15.881490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.881396    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" event={"ID":"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b","Type":"ContainerDied","Data":"affd672a9047aa7159582f2bce5793afba807b2efd78fd36ac44a4cabdf871e2"}
Nov 03 21:08:15.884336 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.884296    5915 generic.go:356] "Generic (PLEG): container finished" podID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerID="6148435255fffcc9ebc2972b6f5d6f70e1db57d0250da428ec084bcdba422ef0" exitCode=0
Nov 03 21:08:15.884427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.884384    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" event={"ID":"04b605f4-86d4-4166-a28d-ed1a38fb490f","Type":"ContainerDied","Data":"6148435255fffcc9ebc2972b6f5d6f70e1db57d0250da428ec084bcdba422ef0"}
Nov 03 21:08:15.886147 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.886115    5915 generic.go:356] "Generic (PLEG): container finished" podID="d47c5e43-c802-4ba3-a503-b33787be0662" containerID="bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00" exitCode=0
Nov 03 21:08:15.886322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.886179    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"
Nov 03 21:08:15.886322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.886193    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" event={"ID":"d47c5e43-c802-4ba3-a503-b33787be0662","Type":"ContainerDied","Data":"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"}
Nov 03 21:08:15.886322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.886234    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" event={"ID":"d47c5e43-c802-4ba3-a503-b33787be0662","Type":"ContainerDied","Data":"677feb7c5898316065678ffff211e4c7a444ddd819fae24bbafa8809a1c19dd4"}
Nov 03 21:08:15.886322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.886255    5915 scope.go:122] "RemoveContainer" containerID="bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"
Nov 03 21:08:15.890406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.890065    5915 generic.go:356] "Generic (PLEG): container finished" podID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerID="70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a" exitCode=0
Nov 03 21:08:15.890406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.890161    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" event={"ID":"583e6d5b-ebd6-43e7-a853-863475376c3d","Type":"ContainerDied","Data":"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"}
Nov 03 21:08:15.890406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.890196    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" event={"ID":"583e6d5b-ebd6-43e7-a853-863475376c3d","Type":"ContainerDied","Data":"b9fdfe382ebd64acc78ec7fcd3314d4eccccea12f72b63b4049e9c69b73eb195"}
Nov 03 21:08:15.890406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.890262    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"
Nov 03 21:08:15.957512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.957477    5915 scope.go:122] "RemoveContainer" containerID="bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"
Nov 03 21:08:15.958018 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:15.957991    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00\": container with ID starting with bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00 not found: ID does not exist" containerID="bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"
Nov 03 21:08:15.958097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.958032    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00"} err="failed to get container status \"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00\": rpc error: code = NotFound desc = could not find container \"bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00\": container with ID starting with bcc7b0f755bf8e07ce09b5ae1fa4f629a0886d85b482099da88f7fc0aaa9ad00 not found: ID does not exist"
Nov 03 21:08:15.958097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.958062    5915 scope.go:122] "RemoveContainer" containerID="70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"
Nov 03 21:08:15.988087 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988050    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988119    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988167    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988201    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988423    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988472    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988517    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988601    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.988712 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988689    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.988821 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.988804    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\") pod \"583e6d5b-ebd6-43e7-a853-863475376c3d\" (UID: \"583e6d5b-ebd6-43e7-a853-863475376c3d\") "
Nov 03 21:08:15.989458 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989433    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47c5e43-c802-4ba3-a503-b33787be0662-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.989592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989462    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47c5e43-c802-4ba3-a503-b33787be0662-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.989592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989478    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.989592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989493    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nqmv\" (UniqueName: \"kubernetes.io/projected/d47c5e43-c802-4ba3-a503-b33787be0662-kube-api-access-7nqmv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.989592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989508    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:15.989833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.989759    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.990944 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.990307    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:15.990944 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.990857    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.990944 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.990915    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:15.993029 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:15.992997    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:08:16.004005 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.003962    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.027449 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.027383    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "kube-api-access-8559n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.036606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.035841    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token" pod "583e6d5b-ebd6-43e7-a853-863475376c3d" (UID: "583e6d5b-ebd6-43e7-a853-863475376c3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.040737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.040715    5915 scope.go:122] "RemoveContainer" containerID="70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"
Nov 03 21:08:16.041171 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:16.041145    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a\": container with ID starting with 70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a not found: ID does not exist" containerID="70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"
Nov 03 21:08:16.041214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.041185    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a"} err="failed to get container status \"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a\": rpc error: code = NotFound desc = could not find container \"70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a\": container with ID starting with 70a8890c10b92372d5e55b6f21413a1933d60964c087be5858c0dd818ac15a8a not found: ID does not exist"
Nov 03 21:08:16.055544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.055516    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 21:08:16.071460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.071050    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:16.071460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.071101    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:16.091148 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091116    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091275 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091189    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091346 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091271    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091346 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091311    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091452 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091351    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091550 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091531    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091625    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091715    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.091825 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.091811    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\") pod \"04b605f4-86d4-4166-a28d-ed1a38fb490f\" (UID: \"04b605f4-86d4-4166-a28d-ed1a38fb490f\") "
Nov 03 21:08:16.092305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092280    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.092528 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092504    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8559n\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-kube-api-access-8559n\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092507    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.092630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092536    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092595    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.092630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092615    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092631    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092647    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092662    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092677    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/583e6d5b-ebd6-43e7-a853-863475376c3d-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092694    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/583e6d5b-ebd6-43e7-a853-863475376c3d-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.092869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092706    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.093165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.092897    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.093165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.093060    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:16.095373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.095344    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "kube-api-access-5f7cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.095474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.095450    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.096002 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.095950    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:08:16.096477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.096440    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy" pod "04b605f4-86d4-4166-a28d-ed1a38fb490f" (UID: "04b605f4-86d4-4166-a28d-ed1a38fb490f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.122602 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.114720    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 21:08:16.193142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193112    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193331 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193189    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193331 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193247    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193331 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193280    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193331 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193306    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193336    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193438    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193468    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193545    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.193741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193598    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\") pod \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\" (UID: \"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b\") "
Nov 03 21:08:16.193741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193694    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.193922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.193899    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:16.194059 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194035    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.194137 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194087    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.194579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194537    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194606    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194621    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194636    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194651    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194665    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194680    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.194692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194693    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04b605f4-86d4-4166-a28d-ed1a38fb490f-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194708    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194722    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194738    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f7cc\" (UniqueName: \"kubernetes.io/projected/04b605f4-86d4-4166-a28d-ed1a38fb490f-kube-api-access-5f7cc\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194751    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04b605f4-86d4-4166-a28d-ed1a38fb490f-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.194765    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.195988 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.195962    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:08:16.196082 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.195985    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "kube-api-access-jdp22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.196138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.196092    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:16.196248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.196230    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy" pod "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" (UID: "940ffd46-acfd-4aff-94a6-fcfe07f4bd8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:16.296630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.296591    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.296787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.296636    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.296787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.296651    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.296787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.296665    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdp22\" (UniqueName: \"kubernetes.io/projected/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b-kube-api-access-jdp22\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:16.576760 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.576670    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:16.760589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.759776    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:08:16.760589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.759852    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:08:16.760589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.760318    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 21:08:16.773589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.772436    5915 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:08:16.773589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.772494    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:08:16.795584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.790055    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:16.813888 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.813089    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6kc4j"]
Nov 03 21:08:16.852507 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.852424    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 21:08:16.887474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.887436    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-fn9lv"]
Nov 03 21:08:16.977532 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.977494    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v" event={"ID":"940ffd46-acfd-4aff-94a6-fcfe07f4bd8b","Type":"ContainerDied","Data":"53fbda96f5fd0118dc32aa0b32a61150260d88c4677b30b868a21dc68b86f61a"}
Nov 03 21:08:16.977720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.977594    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"
Nov 03 21:08:16.977720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.977618    5915 scope.go:122] "RemoveContainer" containerID="affd672a9047aa7159582f2bce5793afba807b2efd78fd36ac44a4cabdf871e2"
Nov 03 21:08:16.980398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.980380    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"
Nov 03 21:08:16.980911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.980886    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl" event={"ID":"04b605f4-86d4-4166-a28d-ed1a38fb490f","Type":"ContainerDied","Data":"a82393ec28c326ea6f0725642cd5c27719a7a234f169fb82a444fa2f186bcd8d"}
Nov 03 21:08:16.996003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:16.995880    5915 scope.go:122] "RemoveContainer" containerID="6148435255fffcc9ebc2972b6f5d6f70e1db57d0250da428ec084bcdba422ef0"
Nov 03 21:08:17.071302 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.071267    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 03 21:08:17.071421 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.071323    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 03 21:08:17.151460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.151379    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 21:08:17.159528 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.159498    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-bbn4v"]
Nov 03 21:08:17.280990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.280957    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 21:08:17.286652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.286628    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-clgbl"]
Nov 03 21:08:17.402652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.399591    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:17.683051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.682975    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" path="/var/lib/kubelet/pods/04b605f4-86d4-4166-a28d-ed1a38fb490f/volumes"
Nov 03 21:08:17.683535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.683518    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" path="/var/lib/kubelet/pods/583e6d5b-ebd6-43e7-a853-863475376c3d/volumes"
Nov 03 21:08:17.684038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.684022    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" path="/var/lib/kubelet/pods/940ffd46-acfd-4aff-94a6-fcfe07f4bd8b/volumes"
Nov 03 21:08:17.684793 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.684778    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" path="/var/lib/kubelet/pods/d47c5e43-c802-4ba3-a503-b33787be0662/volumes"
Nov 03 21:08:17.688526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:17.688508    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:08:18.003603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.003552    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:08:18.004223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004208    5915 cpu_manager.go:414] "containerMap: removing container" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy"
Nov 03 21:08:18.004264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004225    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy"
Nov 03 21:08:18.004264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004239    5915 cpu_manager.go:414] "containerMap: removing container" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy"
Nov 03 21:08:18.004264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004245    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy"
Nov 03 21:08:18.004264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004254    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" containerName="maas-api"
Nov 03 21:08:18.004264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004259    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" containerName="maas-api"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004272    5915 cpu_manager.go:414] "containerMap: removing container" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004276    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004298    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004303    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004311    5915 cpu_manager.go:414] "containerMap: removing container" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004315    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004323    5915 cpu_manager.go:414] "containerMap: removing container" podUID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" containerName="maas-api"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004327    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" containerName="maas-api"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004341    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f2252259-fa76-4445-8247-4662de7f647d" containerName="payload-pre-processing"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004346    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2252259-fa76-4445-8247-4662de7f647d" containerName="payload-pre-processing"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004354    5915 cpu_manager.go:414] "containerMap: removing container" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004359    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004371    5915 cpu_manager.go:414] "containerMap: removing container" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004376    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004383    5915 cpu_manager.go:414] "containerMap: removing container" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004387    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004485    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2252259-fa76-4445-8247-4662de7f647d" containerName="payload-pre-processing"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004501    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004509    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="04b605f4-86d4-4166-a28d-ed1a38fb490f" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004518    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="940ffd46-acfd-4aff-94a6-fcfe07f4bd8b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004529    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="649ea92f-4fd0-4e09-8299-8977c6383d1a" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004537    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="d47c5e43-c802-4ba3-a503-b33787be0662" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004543    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="583e6d5b-ebd6-43e7-a853-863475376c3d" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004547    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="5318bf4b-8fc2-42c9-b540-35e2ac66ce98" containerName="maas-api"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004568    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004579    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7175c94c-c24a-4912-a86f-f0c9b97ac5ec" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004586    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="37882b8d-ba0a-41b3-b1d6-f8c896248387" containerName="istio-proxy"
Nov 03 21:08:18.005018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.004592    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ac4a820-85e1-42a6-8f60-b716aa29eb81" containerName="maas-api"
Nov 03 21:08:18.009277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.009262    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.087632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.083100    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-2f36ee-openshift-default-dockercfg-wnxp9\""
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169549    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169640    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169711    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtqxq\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169783    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169880    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.169948    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.170000    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.170027    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.170066    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.194625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.193484    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:18.263676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.210357    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271346    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271426    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271469    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271635    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271684    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271758    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtqxq\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271830    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.271934    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.272002    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.273081    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.273683    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.278003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.277230    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.305514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.278144    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.305514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.278528    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.305514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.299416    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.305514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.299884    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:08:18.359628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.336527    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:08:18.389582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.362281    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.389582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.373144    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.389582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.373965    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:18.450939 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.450882    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtqxq\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\") pod \"e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") " pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.532913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.532798    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:18.721908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.721876    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:18.733708 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:18.733673    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:19.193834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.193805    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:08:19.195190 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:19.195155    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9a34e6_a541_4cba_b714_a76243c87f38.slice/crio-062fec9530ddf16b3a5c66129c69ecb2a5a64e607b191a329d4e9c98a7c869fb WatchSource:0}: Error finding container 062fec9530ddf16b3a5c66129c69ecb2a5a64e607b191a329d4e9c98a7c869fb: Status 404 returned error can't find the container with id 062fec9530ddf16b3a5c66129c69ecb2a5a64e607b191a329d4e9c98a7c869fb
Nov 03 21:08:19.197957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.197919    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:19.198032 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.198009    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:19.198077 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.198049    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:19.647756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.647727    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:08:19.647949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.647909    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" podUID="ca511d6f-6e55-4c78-9035-41cffa20ec65" containerName="maas-api" containerID="cri-o://790141e67bbb0581b47a750575d03b11b5bc3481d83e20a19ebc60b80f1ecf13" gracePeriod=30
Nov 03 21:08:19.755634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:19.755597    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:20.044637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.039311    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerStarted","Data":"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"}
Nov 03 21:08:20.044637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.039357    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerStarted","Data":"062fec9530ddf16b3a5c66129c69ecb2a5a64e607b191a329d4e9c98a7c869fb"}
Nov 03 21:08:20.044637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.041948    5915 generic.go:356] "Generic (PLEG): container finished" podID="ca511d6f-6e55-4c78-9035-41cffa20ec65" containerID="790141e67bbb0581b47a750575d03b11b5bc3481d83e20a19ebc60b80f1ecf13" exitCode=0
Nov 03 21:08:20.044637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.042080    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" event={"ID":"ca511d6f-6e55-4c78-9035-41cffa20ec65","Type":"ContainerDied","Data":"790141e67bbb0581b47a750575d03b11b5bc3481d83e20a19ebc60b80f1ecf13"}
Nov 03 21:08:20.073512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.073491    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:08:20.213589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.208121    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\") pod \"ca511d6f-6e55-4c78-9035-41cffa20ec65\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") "
Nov 03 21:08:20.213589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.208305    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\" (UniqueName: \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\") pod \"ca511d6f-6e55-4c78-9035-41cffa20ec65\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") "
Nov 03 21:08:20.213589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.208378    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\") pod \"ca511d6f-6e55-4c78-9035-41cffa20ec65\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") "
Nov 03 21:08:20.213589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.208449    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") pod \"ca511d6f-6e55-4c78-9035-41cffa20ec65\" (UID: \"ca511d6f-6e55-4c78-9035-41cffa20ec65\") "
Nov 03 21:08:20.226640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.222766    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca" pod "ca511d6f-6e55-4c78-9035-41cffa20ec65" (UID: "ca511d6f-6e55-4c78-9035-41cffa20ec65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:20.256271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.255769    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs" pod "ca511d6f-6e55-4c78-9035-41cffa20ec65" (UID: "ca511d6f-6e55-4c78-9035-41cffa20ec65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:20.257962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.257708    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv" pod "ca511d6f-6e55-4c78-9035-41cffa20ec65" (UID: "ca511d6f-6e55-4c78-9035-41cffa20ec65"). InnerVolumeSpecName "kube-api-access-6wrdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:20.259124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.259100    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls" pod "ca511d6f-6e55-4c78-9035-41cffa20ec65" (UID: "ca511d6f-6e55-4c78-9035-41cffa20ec65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:20.310191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.310107    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca511d6f-6e55-4c78-9035-41cffa20ec65-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:20.310191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.310142    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wrdv\" (UniqueName: \"kubernetes.io/projected/ca511d6f-6e55-4c78-9035-41cffa20ec65-kube-api-access-6wrdv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:20.310191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.310153    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:20.310191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.310162    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca511d6f-6e55-4c78-9035-41cffa20ec65-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:20.399596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.396391    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podStartSLOduration=3.396369307 podStartE2EDuration="3.396369307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:20.25299347 +0000 UTC m=+1714.704362648" watchObservedRunningTime="2026-11-03 21:08:20.396369307 +0000 UTC m=+1714.847738485"
Nov 03 21:08:20.734515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.734477    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:20.742442 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:20.742418    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:21.052962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.052885    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"
Nov 03 21:08:21.052962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.052908    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj" event={"ID":"ca511d6f-6e55-4c78-9035-41cffa20ec65","Type":"ContainerDied","Data":"e990391b7ad10985c2b7223c827b0fc7b42560c1f51054f5e246fd2d46f2f476"}
Nov 03 21:08:21.053165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.052976    5915 scope.go:122] "RemoveContainer" containerID="790141e67bbb0581b47a750575d03b11b5bc3481d83e20a19ebc60b80f1ecf13"
Nov 03 21:08:21.053297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.053262    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:21.054468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.054413    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:08:21.220162 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.220130    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:08:21.223832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.223803    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04f5216c-86f59f8758-s5thj"]
Nov 03 21:08:21.598905 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.598840    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:21.682784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.682754    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca511d6f-6e55-4c78-9035-41cffa20ec65" path="/var/lib/kubelet/pods/ca511d6f-6e55-4c78-9035-41cffa20ec65/volumes"
Nov 03 21:08:21.828422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.828386    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:21.999594 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:21.997626    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:22.674229 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:22.674190    5915 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" secret="" err="secret \"e2e-ait-04f5216c-openshift-default-dockercfg-9gzx4\" not found"
Nov 03 21:08:22.706994 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:22.706966    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:08:22.713841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:22.713810    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:23.069222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:23.069187    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy" containerID="cri-o://003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd" gracePeriod=30
Nov 03 21:08:23.534448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:23.534400    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:23.690967 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:23.690925    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:23.863027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:23.862926    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:24.089338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.089302    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:24.447204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.447161    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:08:24.448142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.448120    5915 cpu_manager.go:414] "containerMap: removing container" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:24.448210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.448146    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="368f8fcf-b02e-4e3b-b5a9-20ca404d3f3b" containerName="istio-proxy"
Nov 03 21:08:24.448245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.448212    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ca511d6f-6e55-4c78-9035-41cffa20ec65" containerName="maas-api"
Nov 03 21:08:24.448245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.448219    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca511d6f-6e55-4c78-9035-41cffa20ec65" containerName="maas-api"
Nov 03 21:08:24.448417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.448405    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca511d6f-6e55-4c78-9035-41cffa20ec65" containerName="maas-api"
Nov 03 21:08:24.454516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.454101    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.506810 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.506783    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-70fe43-openshift-default-dockercfg-s665s\""
Nov 03 21:08:24.520469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.520437    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:08:24.528120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.528090    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:24.540437 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540413    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540453    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540629    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540700    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540725    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540743    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.540980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.540961    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.541044 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.541018    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbdfm\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.541121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.541107    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667431    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667496    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667542    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667645    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667695    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667732    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.667924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667756    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.667960    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.668020    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbdfm\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.669084    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.669392    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.669677    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.677794    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.682224    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.683024    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.688831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.684280    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.774596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.773817    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:24.813753 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.813717    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbdfm\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:24.819312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:24.819282    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\") pod \"e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") " pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:25.072689 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:25.072607    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:25.508008 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:25.507981    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:08:25.508390 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:25.508367    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93500355_da00_4f11_b925_11b8553bd9e6.slice/crio-7109e39af32746c16d69cdbfe2e148ad022cb9b9aca676b8310baf36d820830b WatchSource:0}: Error finding container 7109e39af32746c16d69cdbfe2e148ad022cb9b9aca676b8310baf36d820830b: Status 404 returned error can't find the container with id 7109e39af32746c16d69cdbfe2e148ad022cb9b9aca676b8310baf36d820830b
Nov 03 21:08:25.510938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:25.510909    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:25.511018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:25.510969    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:25.511064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:25.511026    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:26.166248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:26.166212    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerStarted","Data":"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"}
Nov 03 21:08:26.166655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:26.166268    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerStarted","Data":"7109e39af32746c16d69cdbfe2e148ad022cb9b9aca676b8310baf36d820830b"}
Nov 03 21:08:27.073332 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.073304    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:27.078631 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.078610    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:27.173957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.173926    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:27.174866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.174845    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:08:27.281501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.281456    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podStartSLOduration=3.281444816 podStartE2EDuration="3.281444816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:26.384628038 +0000 UTC m=+1720.835997214" watchObservedRunningTime="2026-11-03 21:08:27.281444816 +0000 UTC m=+1721.732813992"
Nov 03 21:08:27.302731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:27.302701    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:28.374860 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.374837    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:08:28.437257 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437233    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437269    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437305    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437405    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437443    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437512    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437598    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437634    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.437711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437667    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:28.437855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437702    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:28.437990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.437954    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:28.438134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.438091    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:28.438489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.438469    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.438593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.438496    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.438593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.438511    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.438593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.438527    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.439820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.439797    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "kube-api-access-cfqmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:28.439937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.439876    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:28.440167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.440143    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:08:28.440167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.440157    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:28.539348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539319    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\") pod \"02728763-5797-4fa7-b301-1de0a913edcf\" (UID: \"02728763-5797-4fa7-b301-1de0a913edcf\") "
Nov 03 21:08:28.539704 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539680    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert" pod "02728763-5797-4fa7-b301-1de0a913edcf" (UID: "02728763-5797-4fa7-b301-1de0a913edcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:28.539808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539796    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02728763-5797-4fa7-b301-1de0a913edcf-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.539846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539811    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.539846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539819    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02728763-5797-4fa7-b301-1de0a913edcf-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.539846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539831    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfqmm\" (UniqueName: \"kubernetes.io/projected/02728763-5797-4fa7-b301-1de0a913edcf-kube-api-access-cfqmm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.539846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.539840    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02728763-5797-4fa7-b301-1de0a913edcf-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:28.738933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:28.738888    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:29.002646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.002516    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:29.172579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.171832    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173024    5915 cpu_manager.go:414] "containerMap: removing container" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173046    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173145    5915 cpu_manager.go:414] "containerMap: removing container" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173157    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173346    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.176580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.173370    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="02728763-5797-4fa7-b301-1de0a913edcf" containerName="istio-proxy"
Nov 03 21:08:29.197096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.182059    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.208355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.208309    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.210053 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.210026    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dwdf\" (UniqueName: \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.220161 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.220124    5915 generic.go:356] "Generic (PLEG): container finished" podID="02728763-5797-4fa7-b301-1de0a913edcf" containerID="003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd" exitCode=0
Nov 03 21:08:29.222665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.222646    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"
Nov 03 21:08:29.241881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.241603    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-2f36ee\""
Nov 03 21:08:29.242382 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.242308    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-2f36ee-dockercfg-mxbzp\""
Nov 03 21:08:29.242677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.242545    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerDied","Data":"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"}
Nov 03 21:08:29.242857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.242654    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58" event={"ID":"02728763-5797-4fa7-b301-1de0a913edcf","Type":"ContainerDied","Data":"f2537404cd4fb151cf6e1ae64982e2df56ff45c6d79e1ff941ef473c80200bc1"}
Nov 03 21:08:29.242857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.242818    5915 scope.go:122] "RemoveContainer" containerID="003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"
Nov 03 21:08:29.275174 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.275090    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:08:29.340415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.340378    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:08:29.371633 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.345506    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dwdf\" (UniqueName: \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.371633 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.345795    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.433641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.432106    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.461357 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.461235    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dwdf\" (UniqueName: \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\") pod \"payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.525787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.525709    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:29.542624 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.541437    5915 scope.go:122] "RemoveContainer" containerID="5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"
Nov 03 21:08:29.546997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.546965    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:08:29.547103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.547089    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.562136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.562103    5915 scope.go:122] "RemoveContainer" containerID="003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"
Nov 03 21:08:29.562501 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:29.562472    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd\": container with ID starting with 003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd not found: ID does not exist" containerID="003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"
Nov 03 21:08:29.562636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.562515    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd"} err="failed to get container status \"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd\": rpc error: code = NotFound desc = could not find container \"003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd\": container with ID starting with 003c606f4f5939bcfd987236e5da8459283a4f5de58ca3fad5531cd9c0eefdcd not found: ID does not exist"
Nov 03 21:08:29.562636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.562539    5915 scope.go:122] "RemoveContainer" containerID="5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"
Nov 03 21:08:29.562874 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:29.562848    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841\": container with ID starting with 5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841 not found: ID does not exist" containerID="5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"
Nov 03 21:08:29.562935 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.562882    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841"} err="failed to get container status \"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841\": rpc error: code = NotFound desc = could not find container \"5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841\": container with ID starting with 5494cef0393e29d54686216e9111a8b7a8c277ff136b21065a182fb8d231f841 not found: ID does not exist"
Nov 03 21:08:29.639711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.639495    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:08:29.645908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.645631    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-04f5216c-openshift-default-95d6b7759-d8x58"]
Nov 03 21:08:29.654326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.654208    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-276r8\" (UniqueName: \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.654326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.654279    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.684643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.684614    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02728763-5797-4fa7-b301-1de0a913edcf" path="/var/lib/kubelet/pods/02728763-5797-4fa7-b301-1de0a913edcf/volumes"
Nov 03 21:08:29.756125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.756082    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-276r8\" (UniqueName: \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.756267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.756201    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.757239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.757216    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.824520 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.824444    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-276r8\" (UniqueName: \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\") pod \"payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.866788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.866757    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:29.930969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.930940    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 21:08:29.941189 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:29.941160    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03bbb487_1cfa_47c0_ba3e_91579d7ef660.slice/crio-ba429c6e7694fdc4437f8ed41cac69c2e930967f68c971d5af0c4407932fc0c9 WatchSource:0}: Error finding container ba429c6e7694fdc4437f8ed41cac69c2e930967f68c971d5af0c4407932fc0c9: Status 404 returned error can't find the container with id ba429c6e7694fdc4437f8ed41cac69c2e930967f68c971d5af0c4407932fc0c9
Nov 03 21:08:29.942498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:29.942471    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:08:30.033006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.032969    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 21:08:30.241754 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.241710    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" event={"ID":"03bbb487-1cfa-47c0-ba3e-91579d7ef660","Type":"ContainerStarted","Data":"572628db07ccc903ab33fdf2c3846ba8c85f192e3f13902464cab51857ea8c48"}
Nov 03 21:08:30.241929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.241815    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" event={"ID":"03bbb487-1cfa-47c0-ba3e-91579d7ef660","Type":"ContainerStarted","Data":"ba429c6e7694fdc4437f8ed41cac69c2e930967f68c971d5af0c4407932fc0c9"}
Nov 03 21:08:30.242891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.242868    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:30.250718 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:30.250685    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8a6173_0413_453f_a8ff_ae42cd0871bf.slice/crio-3f511208eac3c0ec4953a584fb2c08e2c46fda9465e42258cfb7cfefee845113 WatchSource:0}: Error finding container 3f511208eac3c0ec4953a584fb2c08e2c46fda9465e42258cfb7cfefee845113: Status 404 returned error can't find the container with id 3f511208eac3c0ec4953a584fb2c08e2c46fda9465e42258cfb7cfefee845113
Nov 03 21:08:30.252450 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.252418    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:08:30.298420 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.298391    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:30.432601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.423651    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" podStartSLOduration=1.42362719 podStartE2EDuration="1.42362719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:30.359004668 +0000 UTC m=+1724.810373858" watchObservedRunningTime="2026-11-03 21:08:30.42362719 +0000 UTC m=+1724.874996369"
Nov 03 21:08:30.457404 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.457292    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:30.653644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:30.653519    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:31.253516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:31.253481    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" event={"ID":"ea8a6173-0413-453f-a8ff-ae42cd0871bf","Type":"ContainerStarted","Data":"959187b8b20ea8d0cadb8115421075c7e996af0f12dd3123ad5a9506fbafaeda"}
Nov 03 21:08:31.253715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:31.253538    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" event={"ID":"ea8a6173-0413-453f-a8ff-ae42cd0871bf","Type":"ContainerStarted","Data":"3f511208eac3c0ec4953a584fb2c08e2c46fda9465e42258cfb7cfefee845113"}
Nov 03 21:08:31.253715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:31.253604    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:31.382338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:31.382277    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" podStartSLOduration=2.382264095 podStartE2EDuration="2.382264095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:31.379552064 +0000 UTC m=+1725.830921242" watchObservedRunningTime="2026-11-03 21:08:31.382264095 +0000 UTC m=+1725.833633271"
Nov 03 21:08:31.523646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:31.522940    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:33.129444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:33.129392    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:34.931638 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:34.931586    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing" containerID="cri-o://54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642" gracePeriod=25
Nov 03 21:08:35.035139 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:35.035093    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing" containerID="cri-o://8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b" gracePeriod=25
Nov 03 21:08:37.005832 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:37.005792    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:08:37.397119 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:37.397011    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:37.854043 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:37.854014    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:38.271095 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.271051    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:38.523638 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.523203    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:08:38.566581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.565851    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.610581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.605689    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623602    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623646    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623705    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623748    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623807    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnnds\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.623871    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.624000    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.624039    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.626062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.624098    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.643267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.634167    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-218976-openshift-default-dockercfg-fk98t\""
Nov 03 21:08:38.729066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.724548    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771097    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnnds\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771172    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771281    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771312    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771357    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771476    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771496    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771541    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.771597    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.779641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.772174    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.925583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.918833    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.933640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.930590    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:38.933821 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.933699    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.934505 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.934374    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.936918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.936896    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.985625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.963754    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.985625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.965251    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.993287 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.993260    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnnds\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:38.997627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:38.997603    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\") pod \"e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") " pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:39.117823 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.113698    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:39.191510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.187842    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:08:39.208662 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.208633    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.239875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.239697    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpktt\" (UniqueName: \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.239875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.239867    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.257478 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.256651    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:08:39.261175 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.261143    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-70fe43-dockercfg-rw6vv\""
Nov 03 21:08:39.261305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.261192    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-70fe43\""
Nov 03 21:08:39.268497 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.268476    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:39.280190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.280159    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:39.322141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.322099    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:08:39.344658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.344593    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.344906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.344858    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpktt\" (UniqueName: \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.345467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.345443    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.354769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.354742    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.390547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.389614    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:08:39.413999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.413966    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpktt\" (UniqueName: \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\") pod \"payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.528587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.525735    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.528587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.526030    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64nsw\" (UniqueName: \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.569510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.568829    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:39.644628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.641822    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.644628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.642062    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64nsw\" (UniqueName: \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.644628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.643540    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.703489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.703461    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64nsw\" (UniqueName: \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\") pod \"payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:39.820126 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.820094    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:08:39.821188 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:39.821164    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfeabfcb_aa53_41fd_861a_a4ab990de3a4.slice/crio-b9df0854a9623617498f744c9a0dced9ed9056dc5b5e6e1d08070e97790a178b WatchSource:0}: Error finding container b9df0854a9623617498f744c9a0dced9ed9056dc5b5e6e1d08070e97790a178b: Status 404 returned error can't find the container with id b9df0854a9623617498f744c9a0dced9ed9056dc5b5e6e1d08070e97790a178b
Nov 03 21:08:39.823987 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.823948    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:39.824078 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.824031    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:39.824078 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.824064    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:39.974733 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:39.974693    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:40.068610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.067732    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:08:40.162705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.162675    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:08:40.168208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.168183    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.232110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.232048    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:08:40.237232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.237206    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:40.252384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252350    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252551 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252427    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252551 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252458    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252585    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252631    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252684    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252776    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252805    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xp24\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.252914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.252876    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.298605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.295840    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-b8ac68-openshift-default-dockercfg-ftnbg\""
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359590    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359671    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359703    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359814    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359860    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359916    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.359976    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.360002    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xp24\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.360073    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.360989    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.361698    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.362716    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.362850    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.365646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.362990    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.377595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.371255    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.392290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.388178    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.437105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.434330    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.438918 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:40.438317    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49ae2ca8_3685_4a42_b996_5f408a0f1942.slice/crio-1ee25ef709fa88fd3dfdedd6ad82fe80ac387bff5483902bdfc0be8b93a4a461 WatchSource:0}: Error finding container 1ee25ef709fa88fd3dfdedd6ad82fe80ac387bff5483902bdfc0be8b93a4a461: Status 404 returned error can't find the container with id 1ee25ef709fa88fd3dfdedd6ad82fe80ac387bff5483902bdfc0be8b93a4a461
Nov 03 21:08:40.438918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.438531    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:40.457290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.457259    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerStarted","Data":"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"}
Nov 03 21:08:40.457501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.457480    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerStarted","Data":"b9df0854a9623617498f744c9a0dced9ed9056dc5b5e6e1d08070e97790a178b"}
Nov 03 21:08:40.460853 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.460814    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" event={"ID":"5517b19e-6acb-4b24-8525-18fa2d9ee8a5","Type":"ContainerStarted","Data":"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"}
Nov 03 21:08:40.461044 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.461028    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" event={"ID":"5517b19e-6acb-4b24-8525-18fa2d9ee8a5","Type":"ContainerStarted","Data":"79adc7d2efc492e4b499a554b8607b7b90342b7fc1dec73ba4f8e28fdcc87d27"}
Nov 03 21:08:40.461936 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.461917    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:40.494582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.487631    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xp24\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\") pod \"e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") " pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.494582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.488918    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:40.523889 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.501670    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:08:40.597369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.597264    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podStartSLOduration=2.597243309 podStartE2EDuration="2.597243309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:40.584429571 +0000 UTC m=+1735.035798750" watchObservedRunningTime="2026-11-03 21:08:40.597243309 +0000 UTC m=+1735.048612489"
Nov 03 21:08:40.742588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.712208    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" podStartSLOduration=1.712188531 podStartE2EDuration="1.712188531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:40.710462948 +0000 UTC m=+1735.161832136" watchObservedRunningTime="2026-11-03 21:08:40.712188531 +0000 UTC m=+1735.163557714"
Nov 03 21:08:40.749016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.748932    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:40.880592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.877732    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:08:40.883849 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:40.883806    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:08:41.098166 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.098124    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:41.269697 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.269660    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:41.276097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.276070    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:41.358009 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.357976    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:08:41.358914 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:41.358866    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb672755_dd4e_430d_af31_4da8f757e02c.slice/crio-19acdfffdbd70d6f204a92f71706aac6df5f6e509e4367b5a48bccbe2a3b916b WatchSource:0}: Error finding container 19acdfffdbd70d6f204a92f71706aac6df5f6e509e4367b5a48bccbe2a3b916b: Status 404 returned error can't find the container with id 19acdfffdbd70d6f204a92f71706aac6df5f6e509e4367b5a48bccbe2a3b916b
Nov 03 21:08:41.362757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.362719    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:41.362867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.362799    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:41.362867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.362845    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:08:41.475942 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.475148    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" event={"ID":"49ae2ca8-3685-4a42-b996-5f408a0f1942","Type":"ContainerStarted","Data":"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"}
Nov 03 21:08:41.475942 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.475206    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" event={"ID":"49ae2ca8-3685-4a42-b996-5f408a0f1942","Type":"ContainerStarted","Data":"1ee25ef709fa88fd3dfdedd6ad82fe80ac387bff5483902bdfc0be8b93a4a461"}
Nov 03 21:08:41.475942 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.475456    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:41.479765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.478683    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerStarted","Data":"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"}
Nov 03 21:08:41.479765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.478780    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerStarted","Data":"19acdfffdbd70d6f204a92f71706aac6df5f6e509e4367b5a48bccbe2a3b916b"}
Nov 03 21:08:41.479765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.479348    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:41.480316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.480297    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:08:41.489528 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.489502    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:41.621240 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.621096    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" podStartSLOduration=2.621073234 podStartE2EDuration="2.621073234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:41.593070973 +0000 UTC m=+1736.044440162" watchObservedRunningTime="2026-11-03 21:08:41.621073234 +0000 UTC m=+1736.072442412"
Nov 03 21:08:41.808914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.808879    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:08:41.809186 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.809151    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:08:41.987714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:41.987625    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podStartSLOduration=1.97410983 podStartE2EDuration="1.97410983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:41.863281622 +0000 UTC m=+1736.314650800" watchObservedRunningTime="2026-11-03 21:08:41.97410983 +0000 UTC m=+1736.425479009"
Nov 03 21:08:42.330537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:42.330450    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:08:42.331977 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:42.331937    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:08:42.505599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:42.505039    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:42.505599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:42.505094    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:42.605637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:42.530255    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:08:43.716214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:43.716170    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:43.892434 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:43.892404    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:44.080641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:44.076676    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:44.641446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:44.641413    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:44.902905 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:44.902821    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:45.090046 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:45.090012    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:47.012009 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.011981    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:47.202335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.202300    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:47.508450 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.500652    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:08:47.575596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.570785    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.602638 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.596023    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:08:47.747583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.636450    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-218976-dockercfg-lw75g\""
Nov 03 21:08:47.757019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.647243    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-218976\""
Nov 03 21:08:47.757719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.686947    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrdp5\" (UniqueName: \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.795743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.785265    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.838369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.838342    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:08:47.846925 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.846902    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:08:47.847260 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.847245    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:47.886472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.886444    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2km2\" (UniqueName: \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:47.886677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.886657    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:47.886739 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.886698    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrdp5\" (UniqueName: \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.886883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.886863    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.887728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.887706    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.947967 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.947944    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrdp5\" (UniqueName: \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\") pod \"payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:47.987636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.987612    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:47.988102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.987814    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2km2\" (UniqueName: \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:47.988391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:47.988373    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:48.050190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.050121    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2km2\" (UniqueName: \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\") pod \"payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:48.160434 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.160401    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:48.171641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.171619    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:48.541336 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.541305    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:08:48.541439 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:48.541400    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6c3ae2_33be_457a_8177_092699dcd3fa.slice/crio-45e7086cc6c2d3207dc9b7ecabdb271f5155758a0b2afb276b09623d401e3e7e WatchSource:0}: Error finding container 45e7086cc6c2d3207dc9b7ecabdb271f5155758a0b2afb276b09623d401e3e7e: Status 404 returned error can't find the container with id 45e7086cc6c2d3207dc9b7ecabdb271f5155758a0b2afb276b09623d401e3e7e
Nov 03 21:08:48.590636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.590614    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:08:48.593842 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:48.593816    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2ddfbe_f153_4ab2_9bc1_36aaa7f48b1e.slice/crio-b58add992769e60c26e84d4d7da9cd005baa02774eb30d05be426b700832be0e WatchSource:0}: Error finding container b58add992769e60c26e84d4d7da9cd005baa02774eb30d05be426b700832be0e: Status 404 returned error can't find the container with id b58add992769e60c26e84d4d7da9cd005baa02774eb30d05be426b700832be0e
Nov 03 21:08:48.786957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.786921    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" event={"ID":"3b6c3ae2-33be-457a-8177-092699dcd3fa","Type":"ContainerStarted","Data":"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"}
Nov 03 21:08:48.787141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.786977    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" event={"ID":"3b6c3ae2-33be-457a-8177-092699dcd3fa","Type":"ContainerStarted","Data":"45e7086cc6c2d3207dc9b7ecabdb271f5155758a0b2afb276b09623d401e3e7e"}
Nov 03 21:08:48.787141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.787009    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:48.788902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.788826    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" event={"ID":"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e","Type":"ContainerStarted","Data":"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"}
Nov 03 21:08:48.788902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.788883    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" event={"ID":"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e","Type":"ContainerStarted","Data":"b58add992769e60c26e84d4d7da9cd005baa02774eb30d05be426b700832be0e"}
Nov 03 21:08:48.789064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.788915    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:48.904948 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:48.904896    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" podStartSLOduration=1.904883211 podStartE2EDuration="1.904883211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:48.901945132 +0000 UTC m=+1743.353314310" watchObservedRunningTime="2026-11-03 21:08:48.904883211 +0000 UTC m=+1743.356252366"
Nov 03 21:08:49.063951 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:49.063840    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" podStartSLOduration=2.063821137 podStartE2EDuration="2.063821137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:49.062796482 +0000 UTC m=+1743.514165659" watchObservedRunningTime="2026-11-03 21:08:49.063821137 +0000 UTC m=+1743.515190316"
Nov 03 21:08:49.327128 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:49.327032    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:50.536118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:50.536076    5915 scope.go:122] "RemoveContainer" containerID="ec05e81dbdf11bfcc6bf3e2c8f12550d2da8816418b95084185f03da5676d70e"
Nov 03 21:08:50.547818 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:50.547798    5915 scope.go:122] "RemoveContainer" containerID="facad53502a73997c58cc9d73011bc589301135176fca1499fa1122ffcb723c9"
Nov 03 21:08:50.856929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:50.856827    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:08:50.856929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:50.856872    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:08:52.501206 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:52.501175    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:08:52.506776 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:52.506754    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:08:54.133600 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.133531    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:54.350182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.350142    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:54.595414 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.590364    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:54.719145 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.719111    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 21:08:54.719624 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.719592    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" containerID="cri-o://d9d968154e9c6b569fec4f3ac63e7187b7466241cd5342364153da5c7df094f8" gracePeriod=30
Nov 03 21:08:54.744742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.744705    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 21:08:54.745603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.745533    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy" containerID="cri-o://6718d7a89115ac2811e62f6a5853632eba462c3eadb19532f4c6226fb4d66ea4" gracePeriod=30
Nov 03 21:08:54.749104 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.749079    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:08:54.749869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.749252    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy" containerID="cri-o://cffb6d455c782b5944c55b924201930c5784d1c2f73207eeabb8f61fe619717c" gracePeriod=30
Nov 03 21:08:54.792769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.792738    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:08:54.792960 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.792941    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy" containerID="cri-o://30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038" gracePeriod=30
Nov 03 21:08:54.799650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.799622    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:08:54.807251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.807231    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:54.851998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.851941    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:08:54.861711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.861685    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-b8ac68\""
Nov 03 21:08:54.862207 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.862187    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-b8ac68-dockercfg-b9cc7\""
Nov 03 21:08:54.909139 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.909108    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw69w\" (UniqueName: \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:54.909318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.909152    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:54.911809 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.911788    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:08:54.916661 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.916639    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:54.974324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:54.974294    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:08:55.041659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.023090    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.042093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.042067    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw69w\" (UniqueName: \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:55.042289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.042271    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:55.042409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.042396    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xw8g\" (UniqueName: \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.063639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.043407    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:55.193351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.136663    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:55.193351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.139441    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw69w\" (UniqueName: \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\") pod \"payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:55.193351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.144465    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.193351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.144617    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xw8g\" (UniqueName: \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.193351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.145875    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.221912 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.214823    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xw8g\" (UniqueName: \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\") pod \"payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.235811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.235779    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.319351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.319315    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:55.423745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.423270    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:55.501240 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.501209    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 21:08:55.564625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.561528    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\" (UniqueName: \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\") pod \"36495e2b-373e-4985-b0e7-fb0b397352b9\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") "
Nov 03 21:08:55.564625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.561824    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\") pod \"36495e2b-373e-4985-b0e7-fb0b397352b9\" (UID: \"36495e2b-373e-4985-b0e7-fb0b397352b9\") "
Nov 03 21:08:55.564625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.562843    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume" pod "36495e2b-373e-4985-b0e7-fb0b397352b9" (UID: "36495e2b-373e-4985-b0e7-fb0b397352b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:55.566456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.566330    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9" pod "36495e2b-373e-4985-b0e7-fb0b397352b9" (UID: "36495e2b-373e-4985-b0e7-fb0b397352b9"). InnerVolumeSpecName "kube-api-access-h6tf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:55.585047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.585011    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 21:08:55.662941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.662859    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\" (UniqueName: \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\") pod \"0fccc01f-9b8b-42d7-8c81-765059c9f791\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") "
Nov 03 21:08:55.663098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.663084    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\") pod \"0fccc01f-9b8b-42d7-8c81-765059c9f791\" (UID: \"0fccc01f-9b8b-42d7-8c81-765059c9f791\") "
Nov 03 21:08:55.663459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.663434    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume" pod "0fccc01f-9b8b-42d7-8c81-765059c9f791" (UID: "0fccc01f-9b8b-42d7-8c81-765059c9f791"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:08:55.663616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.663599    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36495e2b-373e-4985-b0e7-fb0b397352b9-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:55.663679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.663621    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6tf9\" (UniqueName: \"kubernetes.io/projected/36495e2b-373e-4985-b0e7-fb0b397352b9-kube-api-access-h6tf9\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:55.663679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.663635    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fccc01f-9b8b-42d7-8c81-765059c9f791-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:55.665096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.665074    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf" pod "0fccc01f-9b8b-42d7-8c81-765059c9f791" (UID: "0fccc01f-9b8b-42d7-8c81-765059c9f791"). InnerVolumeSpecName "kube-api-access-h85xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:55.694124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.694100    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:08:55.697059 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:55.697039    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296c205e_4ebf_4b9c_be50_1b9b98ea5598.slice/crio-e2d49f058ae91aa67166dbb65fb56d2dbef68bb99f198be6fb0a428a90f506a9 WatchSource:0}: Error finding container e2d49f058ae91aa67166dbb65fb56d2dbef68bb99f198be6fb0a428a90f506a9: Status 404 returned error can't find the container with id e2d49f058ae91aa67166dbb65fb56d2dbef68bb99f198be6fb0a428a90f506a9
Nov 03 21:08:55.765290 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.765259    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h85xf\" (UniqueName: \"kubernetes.io/projected/0fccc01f-9b8b-42d7-8c81-765059c9f791-kube-api-access-h85xf\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:55.838535 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:08:55.838507    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6678936_ca03_4f7a_888d_23bb2cb87cb4.slice/crio-abd4f2cc48999ad3e9258876b3a1ceac988f836db8aa8828c70bd5ca0c2faab8 WatchSource:0}: Error finding container abd4f2cc48999ad3e9258876b3a1ceac988f836db8aa8828c70bd5ca0c2faab8: Status 404 returned error can't find the container with id abd4f2cc48999ad3e9258876b3a1ceac988f836db8aa8828c70bd5ca0c2faab8
Nov 03 21:08:55.838682 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.838603    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:08:55.844669 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.844648    5915 generic.go:356] "Generic (PLEG): container finished" podID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerID="54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642" exitCode=0
Nov 03 21:08:55.844759 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.844714    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"
Nov 03 21:08:55.844759 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.844728    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" event={"ID":"0fccc01f-9b8b-42d7-8c81-765059c9f791","Type":"ContainerDied","Data":"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"}
Nov 03 21:08:55.844873 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.844775    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d" event={"ID":"0fccc01f-9b8b-42d7-8c81-765059c9f791","Type":"ContainerDied","Data":"62995504ae6b196b987e79bfde21438f4ad4844bd9b77919d7597a6883cc0ce4"}
Nov 03 21:08:55.844873 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.844804    5915 scope.go:122] "RemoveContainer" containerID="54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"
Nov 03 21:08:55.847130 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.847102    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" event={"ID":"296c205e-4ebf-4b9c-be50-1b9b98ea5598","Type":"ContainerStarted","Data":"9736ece8fb0090d83fc153c3d6e530d4addce34ae6ba17f6e95266111318e34a"}
Nov 03 21:08:55.847239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.847160    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" event={"ID":"296c205e-4ebf-4b9c-be50-1b9b98ea5598","Type":"ContainerStarted","Data":"e2d49f058ae91aa67166dbb65fb56d2dbef68bb99f198be6fb0a428a90f506a9"}
Nov 03 21:08:55.847239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.847226    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:08:55.848897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.848875    5915 generic.go:356] "Generic (PLEG): container finished" podID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerID="8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b" exitCode=0
Nov 03 21:08:55.848986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.848917    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" event={"ID":"36495e2b-373e-4985-b0e7-fb0b397352b9","Type":"ContainerDied","Data":"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"}
Nov 03 21:08:55.848986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.848955    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh" event={"ID":"36495e2b-373e-4985-b0e7-fb0b397352b9","Type":"ContainerDied","Data":"5b37461c2935306523c0fd818c9854219847203f41016a4e701ec61041dbc9cd"}
Nov 03 21:08:55.848986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.848919    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"
Nov 03 21:08:55.857014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.856995    5915 scope.go:122] "RemoveContainer" containerID="54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"
Nov 03 21:08:55.857343 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:55.857322    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642\": container with ID starting with 54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642 not found: ID does not exist" containerID="54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"
Nov 03 21:08:55.857403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.857350    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642"} err="failed to get container status \"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642\": rpc error: code = NotFound desc = could not find container \"54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642\": container with ID starting with 54e0109d699b3036260ac82497cb4bb8c1818e00d8812d3655af4808e7fbe642 not found: ID does not exist"
Nov 03 21:08:55.857403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.857367    5915 scope.go:122] "RemoveContainer" containerID="8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"
Nov 03 21:08:55.868908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.868889    5915 scope.go:122] "RemoveContainer" containerID="8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"
Nov 03 21:08:55.869169 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:08:55.869150    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b\": container with ID starting with 8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b not found: ID does not exist" containerID="8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"
Nov 03 21:08:55.869232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:55.869175    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b"} err="failed to get container status \"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b\": rpc error: code = NotFound desc = could not find container \"8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b\": container with ID starting with 8bd30a2180dea301d42985aad94b1aaed6cbd0e761ab2ee5c1708d8bd895502b not found: ID does not exist"
Nov 03 21:08:56.015974 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.015936    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 21:08:56.020724 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.020698    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-89bb58-56d644cf59-r45jh"]
Nov 03 21:08:56.131846 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.131814    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 21:08:56.139640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.139614    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-89bb58-86bf78ccbd-mcx7d"]
Nov 03 21:08:56.252297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.252247    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" podStartSLOduration=2.252234139 podStartE2EDuration="2.252234139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:56.251817426 +0000 UTC m=+1750.703186603" watchObservedRunningTime="2026-11-03 21:08:56.252234139 +0000 UTC m=+1750.703603315"
Nov 03 21:08:56.860354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.860322    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" event={"ID":"b6678936-ca03-4f7a-888d-23bb2cb87cb4","Type":"ContainerStarted","Data":"449e13c54fa1eb11debe9fd7a42c8a6b797ce87930954adf9583ca58dbc0b092"}
Nov 03 21:08:56.860354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.860371    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" event={"ID":"b6678936-ca03-4f7a-888d-23bb2cb87cb4","Type":"ContainerStarted","Data":"abd4f2cc48999ad3e9258876b3a1ceac988f836db8aa8828c70bd5ca0c2faab8"}
Nov 03 21:08:56.860669 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.860654    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:08:56.987006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:56.986955    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" podStartSLOduration=2.986942726 podStartE2EDuration="2.986942726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:08:56.98231415 +0000 UTC m=+1751.433683334" watchObservedRunningTime="2026-11-03 21:08:56.986942726 +0000 UTC m=+1751.438311902"
Nov 03 21:08:57.683027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:57.682997    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" path="/var/lib/kubelet/pods/0fccc01f-9b8b-42d7-8c81-765059c9f791/volumes"
Nov 03 21:08:57.683440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:57.683424    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" path="/var/lib/kubelet/pods/36495e2b-373e-4985-b0e7-fb0b397352b9/volumes"
Nov 03 21:08:58.980874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:58.980843    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:08:59.502312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.502271    5915 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:08:59.502471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.502344    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:08:59.801517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.801489    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:08:59.801738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.801591    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:08:59.889786 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.889757    5915 generic.go:356] "Generic (PLEG): container finished" podID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerID="d9d968154e9c6b569fec4f3ac63e7187b7466241cd5342364153da5c7df094f8" exitCode=0
Nov 03 21:08:59.889907 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.889830    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" event={"ID":"fbbf175c-323a-4c4b-830b-547a3cbee8ad","Type":"ContainerDied","Data":"d9d968154e9c6b569fec4f3ac63e7187b7466241cd5342364153da5c7df094f8"}
Nov 03 21:08:59.891898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.891876    5915 generic.go:356] "Generic (PLEG): container finished" podID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerID="6718d7a89115ac2811e62f6a5853632eba462c3eadb19532f4c6226fb4d66ea4" exitCode=0
Nov 03 21:08:59.891997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.891948    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerDied","Data":"6718d7a89115ac2811e62f6a5853632eba462c3eadb19532f4c6226fb4d66ea4"}
Nov 03 21:08:59.891997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.891994    5915 scope.go:122] "RemoveContainer" containerID="f74adf22cccdd264d4c1eab4dac873779a69c2e4ea7d301e03d45a871c5817da"
Nov 03 21:08:59.893815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.893795    5915 generic.go:356] "Generic (PLEG): container finished" podID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerID="cffb6d455c782b5944c55b924201930c5784d1c2f73207eeabb8f61fe619717c" exitCode=0
Nov 03 21:08:59.893888 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:08:59.893827    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" event={"ID":"cd8e8d79-3820-4df6-b86b-ca93da2d5664","Type":"ContainerDied","Data":"cffb6d455c782b5944c55b924201930c5784d1c2f73207eeabb8f61fe619717c"}
Nov 03 21:09:00.088484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.088458    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:00.164000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.163860    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 21:09:00.224462 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.224434    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 21:09:00.226438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.226415    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:09:00.230464 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.230448    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:09:00.250762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250731    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.250924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250808    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.250924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250832    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.250924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250901    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251070 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250963    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251070 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.250993    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251070 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251023    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251180    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251180    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:09:00.251333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251222    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.251333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251233    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\") pod \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\" (UID: \"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a\") "
Nov 03 21:09:00.251333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251305    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.251462 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251374    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.251598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251539    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.251996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251947    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.251996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251973    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.251996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.251987    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.252223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.252000    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.252223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.252013    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.253601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.253530    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:09:00.253768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.253734    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "kube-api-access-95h9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.253871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.253829    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.254215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.254200    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy" pod "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" (UID: "a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.352878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.352789    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.352878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.352842    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.352878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.352861    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.352929    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.352984    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353010    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353053    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353114    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.353169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353161    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.353471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353106    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.353471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353122    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.353471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353205    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.353471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353358    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.353471 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353448    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.353771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353469    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.353771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353596    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.353771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353675    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.353771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353712    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.353771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353738    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.354086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353782    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.354086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353864    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.354086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354008    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.354086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354058    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354103    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354151    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354180    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354248    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354285    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354339    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\") pod \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\" (UID: \"fbbf175c-323a-4c4b-830b-547a3cbee8ad\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354391    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.354473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354461    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\") pod \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\" (UID: \"cd8e8d79-3820-4df6-b86b-ca93da2d5664\") "
Nov 03 21:09:00.354945 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354506    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\") pod \"ab6080f9-7776-494d-8292-f542b7497164\" (UID: \"ab6080f9-7776-494d-8292-f542b7497164\") "
Nov 03 21:09:00.355461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355435    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355466    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355484    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355499    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95h9h\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-kube-api-access-95h9h\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355518    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355530    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355544    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355576    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.355646 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355588    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.353780    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356078    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "kube-api-access-6tfvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354321    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.354913    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355255    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.355926    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356064    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356141    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:09:00.356716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356272    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356276    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.356716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356338    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.356716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.356547    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.357722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.357653    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.358038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.358006    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:09:00.358344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.358262    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:09:00.358344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.358300    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.358656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.358620    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:00.359070 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.359033    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:09:00.359173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.359140    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.359308 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.359287    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp" pod "fbbf175c-323a-4c4b-830b-547a3cbee8ad" (UID: "fbbf175c-323a-4c4b-830b-547a3cbee8ad"). InnerVolumeSpecName "kube-api-access-x9lmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.359676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.359655    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t" pod "ab6080f9-7776-494d-8292-f542b7497164" (UID: "ab6080f9-7776-494d-8292-f542b7497164"). InnerVolumeSpecName "kube-api-access-gkc4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.359959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.359940    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token" pod "cd8e8d79-3820-4df6-b86b-ca93da2d5664" (UID: "cd8e8d79-3820-4df6-b86b-ca93da2d5664"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:00.457484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457451    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457484    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457493    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457507    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457515    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457527    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9lmp\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-kube-api-access-x9lmp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457540    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457569    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457582    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457595    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkc4t\" (UniqueName: \"kubernetes.io/projected/ab6080f9-7776-494d-8292-f542b7497164-kube-api-access-gkc4t\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457611    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457621    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457629    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457637    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab6080f9-7776-494d-8292-f542b7497164-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457646    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457662    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab6080f9-7776-494d-8292-f542b7497164-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457671    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457685    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457710    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbbf175c-323a-4c4b-830b-547a3cbee8ad-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457719    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tfvd\" (UniqueName: \"kubernetes.io/projected/cd8e8d79-3820-4df6-b86b-ca93da2d5664-kube-api-access-6tfvd\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457729    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd8e8d79-3820-4df6-b86b-ca93da2d5664-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.457883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.457738    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab6080f9-7776-494d-8292-f542b7497164-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:00.903661 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.903624    5915 generic.go:356] "Generic (PLEG): container finished" podID="ab6080f9-7776-494d-8292-f542b7497164" containerID="30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038" exitCode=0
Nov 03 21:09:00.903848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.903666    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"
Nov 03 21:09:00.903848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.903715    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" event={"ID":"ab6080f9-7776-494d-8292-f542b7497164","Type":"ContainerDied","Data":"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"}
Nov 03 21:09:00.903848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.903775    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9" event={"ID":"ab6080f9-7776-494d-8292-f542b7497164","Type":"ContainerDied","Data":"df3463362dda92f416352c2029b31b96ec02373d3869f67ae49ddcb586083c70"}
Nov 03 21:09:00.903848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.903806    5915 scope.go:122] "RemoveContainer" containerID="30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"
Nov 03 21:09:00.905414 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.905389    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"
Nov 03 21:09:00.905532 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.905416    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x" event={"ID":"fbbf175c-323a-4c4b-830b-547a3cbee8ad","Type":"ContainerDied","Data":"50ee6568279f36a03eec840e8e2a88427ee0913fc09b8df04fac30449db550fa"}
Nov 03 21:09:00.907315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.907277    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"
Nov 03 21:09:00.907315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.907307    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl" event={"ID":"a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a","Type":"ContainerDied","Data":"3a880913061f7f5a660a03d15cd85909fbbd89bba7742ab3f051c6b58b1e8c5c"}
Nov 03 21:09:00.909126 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.909106    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"
Nov 03 21:09:00.909246 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.909214    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7" event={"ID":"cd8e8d79-3820-4df6-b86b-ca93da2d5664","Type":"ContainerDied","Data":"4b70461791c97733f6b6dbf2d55616ccdfb2434c939dd9590399d906c1eeeaef"}
Nov 03 21:09:00.918026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.917995    5915 scope.go:122] "RemoveContainer" containerID="30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"
Nov 03 21:09:00.918527 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:00.918499    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038\": container with ID starting with 30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038 not found: ID does not exist" containerID="30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"
Nov 03 21:09:00.918815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.918536    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038"} err="failed to get container status \"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038\": rpc error: code = NotFound desc = could not find container \"30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038\": container with ID starting with 30f34a46e682b457fc7e79c0a6120c42d0f6ce0341f17e5a60e8175deefa9038 not found: ID does not exist"
Nov 03 21:09:00.918815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.918589    5915 scope.go:122] "RemoveContainer" containerID="d9d968154e9c6b569fec4f3ac63e7187b7466241cd5342364153da5c7df094f8"
Nov 03 21:09:00.941219 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.941196    5915 scope.go:122] "RemoveContainer" containerID="6718d7a89115ac2811e62f6a5853632eba462c3eadb19532f4c6226fb4d66ea4"
Nov 03 21:09:00.951727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:00.951704    5915 scope.go:122] "RemoveContainer" containerID="cffb6d455c782b5944c55b924201930c5784d1c2f73207eeabb8f61fe619717c"
Nov 03 21:09:01.072880 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.072807    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 21:09:01.088622 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.088594    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-st8xl"]
Nov 03 21:09:01.195992 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.195960    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:09:01.214249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.214214    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-gnqz7"]
Nov 03 21:09:01.316151 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.316122    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:09:01.319848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.319821    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-p8kn9"]
Nov 03 21:09:01.435500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.435423    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 21:09:01.440007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.439983    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-4xt4x"]
Nov 03 21:09:01.683613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.683580    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" path="/var/lib/kubelet/pods/a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a/volumes"
Nov 03 21:09:01.684088 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.684073    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab6080f9-7776-494d-8292-f542b7497164" path="/var/lib/kubelet/pods/ab6080f9-7776-494d-8292-f542b7497164/volumes"
Nov 03 21:09:01.684516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.684498    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" path="/var/lib/kubelet/pods/cd8e8d79-3820-4df6-b86b-ca93da2d5664/volumes"
Nov 03 21:09:01.685246 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.685230    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" path="/var/lib/kubelet/pods/fbbf175c-323a-4c4b-830b-547a3cbee8ad/volumes"
Nov 03 21:09:01.947728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:01.947699    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:02.773604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:02.773550    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:03.383489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:03.383452    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:03.648469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:03.648382    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:04.012284 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.012248    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:04.025694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.025658    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:09:04.026815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026781    5915 cpu_manager.go:414] "containerMap: removing container" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing"
Nov 03 21:09:04.026815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026809    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026835    5915 cpu_manager.go:414] "containerMap: removing container" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026843    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026881    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026888    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026901    5915 cpu_manager.go:414] "containerMap: removing container" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026931    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026949    5915 cpu_manager.go:414] "containerMap: removing container" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026954    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026961    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy"
Nov 03 21:09:04.026991 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.026965    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027099    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbbf175c-323a-4c4b-830b-547a3cbee8ad" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027111    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027117    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd8e8d79-3820-4df6-b86b-ca93da2d5664" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027127    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fccc01f-9b8b-42d7-8c81-765059c9f791" containerName="payload-processing"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027133    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab6080f9-7776-494d-8292-f542b7497164" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027142    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:04.027516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.027150    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="36495e2b-373e-4985-b0e7-fb0b397352b9" containerName="payload-pre-processing"
Nov 03 21:09:04.033023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.032999    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.088334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.088305    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:09:04.127424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.127387    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.127659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.127636    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.127773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.127704    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5nb\" (UniqueName: \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.127773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.127758    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.132898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.132875    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-2f36ee-serving-cert\""
Nov 03 21:09:04.229536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.229494    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.229737 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.229716    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.229876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.229859    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.229942 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.229913    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5nb\" (UniqueName: \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.230050 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:04.230006    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-serving-cert: secret "maas-api-e2e-shared-a-w5-2f36ee-serving-cert" not found
Nov 03 21:09:04.230117 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:04.230082    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls podName:20f0ed39-f25a-4fe7-8346-c8616e20633d nodeName:}" failed. No retries permitted until 2026-11-03 21:09:04.730062582 +0000 UTC m=+1759.181431755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls") pod "maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" (UID: "20f0ed39-f25a-4fe7-8346-c8616e20633d") : secret "maas-api-e2e-shared-a-w5-2f36ee-serving-cert" not found
Nov 03 21:09:04.230492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.230466    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.232669 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.232649    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.237207 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.237186    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:04.323643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.322381    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5nb\" (UniqueName: \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.405816 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.405776    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:04.560376 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.560333    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:04.742105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.742056    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.744713 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.744688    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:04.951422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:04.951387    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:05.339796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:05.339768    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:09:05.340612 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:09:05.340587    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f0ed39_f25a_4fe7_8346_c8616e20633d.slice/crio-d4f45251f765b38564d9ad192baea2bbe59c5e1379d1969af506240d084d6ed8 WatchSource:0}: Error finding container d4f45251f765b38564d9ad192baea2bbe59c5e1379d1969af506240d084d6ed8: Status 404 returned error can't find the container with id d4f45251f765b38564d9ad192baea2bbe59c5e1379d1969af506240d084d6ed8
Nov 03 21:09:06.006066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:06.006030    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" event={"ID":"20f0ed39-f25a-4fe7-8346-c8616e20633d","Type":"ContainerStarted","Data":"d4f45251f765b38564d9ad192baea2bbe59c5e1379d1969af506240d084d6ed8"}
Nov 03 21:09:06.863486 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:06.863457    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:09:07.008763 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:07.008727    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:09:07.018710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:07.018181    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" event={"ID":"20f0ed39-f25a-4fe7-8346-c8616e20633d","Type":"ContainerStarted","Data":"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"}
Nov 03 21:09:07.018710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:07.018252    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:07.874970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:07.874936    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:09:07.991767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:07.991717    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" podStartSLOduration=4.1272114 podStartE2EDuration="4.99170396s" totalImagesPullingTime="864.49256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:09:07.144507273 +0000 UTC m=+1761.595876450" watchObservedRunningTime="2026-11-03 21:09:07.99170396 +0000 UTC m=+1762.443073160"
Nov 03 21:09:09.249087 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:09.249053    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:09.863857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:09.863826    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:10.053806 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:10.053779    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:13.035988 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.035960    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:09:13.771906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.771859    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:09:13.772654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.772634    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:13.772783 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.772654    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a91eeaf6-da8d-41c0-8ff9-4f2f54dd380a" containerName="istio-proxy"
Nov 03 21:09:13.777699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.777677    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.830975 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.830944    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-70fe43-serving-cert\""
Nov 03 21:09:13.870613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.870533    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.870812 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.870739    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.870812 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.870781    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.870944 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.870820    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m425\" (UniqueName: \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.885303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.885273    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:09:13.972318 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.972286    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.972508 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.972422    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.972508 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.972461    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.972508 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:13.972475    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-serving-cert: secret "maas-api-e2e-shared-a-w6-70fe43-serving-cert" not found
Nov 03 21:09:13.972711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.972514    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m425\" (UniqueName: \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.972711 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:13.972551    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls podName:3a5d712e-7942-46cc-b776-a106bf237a92 nodeName:}" failed. No retries permitted until 2026-11-03 21:09:14.472529008 +0000 UTC m=+1768.923898175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls") pod "maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" (UID: "3a5d712e-7942-46cc-b776-a106bf237a92") : secret "maas-api-e2e-shared-a-w6-70fe43-serving-cert" not found
Nov 03 21:09:13.973344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.973324    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:13.974578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.974528    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:13.975535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:13.975516    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:14.100653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.100540    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m425\" (UniqueName: \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:14.131640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.131610    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:14.278278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.278204    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:14.487305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.487277    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:14.489996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.489968    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:14.693834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:14.693803    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:15.137804 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:15.137776    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:09:15.140613 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:09:15.140544    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5d712e_7942_46cc_b776_a106bf237a92.slice/crio-19f463e4dc1b66c7504cacc012f891889ecf5a849f92dbd8eb58184902f9dbd1 WatchSource:0}: Error finding container 19f463e4dc1b66c7504cacc012f891889ecf5a849f92dbd8eb58184902f9dbd1: Status 404 returned error can't find the container with id 19f463e4dc1b66c7504cacc012f891889ecf5a849f92dbd8eb58184902f9dbd1
Nov 03 21:09:15.174319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:15.174289    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" event={"ID":"3a5d712e-7942-46cc-b776-a106bf237a92","Type":"ContainerStarted","Data":"19f463e4dc1b66c7504cacc012f891889ecf5a849f92dbd8eb58184902f9dbd1"}
Nov 03 21:09:16.186050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:16.186000    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" event={"ID":"3a5d712e-7942-46cc-b776-a106bf237a92","Type":"ContainerStarted","Data":"d35afeb2d5ddc4a64c6a7066b7d1babaa0da5f909e17d7777201c1802251d3c9"}
Nov 03 21:09:16.338709 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:16.338645    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" podStartSLOduration=2.385685269 podStartE2EDuration="3.338623936s" totalImagesPullingTime="952.938667ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:09:16.337902298 +0000 UTC m=+1770.789271466" watchObservedRunningTime="2026-11-03 21:09:16.338623936 +0000 UTC m=+1770.789993176"
Nov 03 21:09:17.087743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:17.087714    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:17.193452 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:17.193423    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:17.694735 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:17.694703    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:19.205717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:19.205668    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:22.817547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:22.817505    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:23.211424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:23.211395    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:09:24.428871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:24.428839    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:29.431176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:29.431119    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:29.839078 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:29.839048    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 21:09:29.972049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:29.972018    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:30.209568 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:30.209531    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:30.369152 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:30.369119    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:31.964310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:31.964274    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:34.418903 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.418874    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:09:34.425109 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.425088    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.477073 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.477044    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-218976-serving-cert\""
Nov 03 21:09:34.533166 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.533128    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:09:34.555278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.555245    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvf59\" (UniqueName: \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.555571 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.555532    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.555666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.555584    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.555756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.555738    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.624649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.624616    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:34.657417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.657382    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvf59\" (UniqueName: \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.657929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.657516    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.657929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.657544    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.657929 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:34.657680    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-serving-cert: secret "maas-api-e2e-shared-b-w1-218976-serving-cert" not found
Nov 03 21:09:34.657929 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:34.657773    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls podName:a3b04b00-64e7-473f-91b7-323e491ede5a nodeName:}" failed. No retries permitted until 2026-11-03 21:09:35.157751179 +0000 UTC m=+1789.609120348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls") pod "maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" (UID: "a3b04b00-64e7-473f-91b7-323e491ede5a") : secret "maas-api-e2e-shared-b-w1-218976-serving-cert" not found
Nov 03 21:09:34.657929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.657840    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.658695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.658670    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.660496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.660463    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.730391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.730353    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvf59\" (UniqueName: \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:34.839667 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.839626    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing" containerID="cri-o://0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee" gracePeriod=25
Nov 03 21:09:34.867304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:34.867266    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:35.026112 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.025999    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:35.167134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.167100    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:35.169790 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.169764    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:35.177740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.177711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:35.339269 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.339194    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:35.724520 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:09:35.724489    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b04b00_64e7_473f_91b7_323e491ede5a.slice/crio-12824a5927869232c396e756cd9d107f81a201e23ceadbf1d15502f54c904e40 WatchSource:0}: Error finding container 12824a5927869232c396e756cd9d107f81a201e23ceadbf1d15502f54c904e40: Status 404 returned error can't find the container with id 12824a5927869232c396e756cd9d107f81a201e23ceadbf1d15502f54c904e40
Nov 03 21:09:35.724978 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:35.724705    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:09:36.508117 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:36.508084    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" event={"ID":"a3b04b00-64e7-473f-91b7-323e491ede5a","Type":"ContainerStarted","Data":"12824a5927869232c396e756cd9d107f81a201e23ceadbf1d15502f54c904e40"}
Nov 03 21:09:36.826438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:36.826361    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:09:37.005957 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:37.005922    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:09:37.519517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:37.519481    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" event={"ID":"a3b04b00-64e7-473f-91b7-323e491ede5a","Type":"ContainerStarted","Data":"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"}
Nov 03 21:09:37.519700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:37.519600    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:37.647204 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:37.647145    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" podStartSLOduration=2.842754688 podStartE2EDuration="3.64712453s" totalImagesPullingTime="804.369842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:09:37.644909818 +0000 UTC m=+1792.096278995" watchObservedRunningTime="2026-11-03 21:09:37.64712453 +0000 UTC m=+1792.098493908"
Nov 03 21:09:39.452773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.452732    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:09:39.457358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.457335    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.528822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.528784    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc65w\" (UniqueName: \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.529001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.528944    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.529001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.528992    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.529129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.529064    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.561371 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.561338    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b8ac68-serving-cert\""
Nov 03 21:09:39.565781 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.565754    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:09:39.630736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.630705    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc65w\" (UniqueName: \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.630917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.630790    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.630917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.630818    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.630917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.630871    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.631097 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:39.630985    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-serving-cert: secret "maas-api-e2e-shared-a-w2-b8ac68-serving-cert" not found
Nov 03 21:09:39.631097 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:39.631051    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls podName:ee97cf83-6b4f-4eb0-ad31-1586a0e02b73 nodeName:}" failed. No retries permitted until 2026-11-03 21:09:40.131029819 +0000 UTC m=+1794.582398974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls") pod "maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" (UID: "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73") : secret "maas-api-e2e-shared-a-w2-b8ac68-serving-cert" not found
Nov 03 21:09:39.631682 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.631617    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.633386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.633362    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.655614 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.655591    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:39.694721 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.694696    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc65w\" (UniqueName: \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:39.810922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:39.810841    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:40.086027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:40.085938    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:40.180930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:40.180899    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:40.183600 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:40.183538    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:40.372094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:40.372008    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:40.752484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:40.752453    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:09:40.753944 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:09:40.753917    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee97cf83_6b4f_4eb0_ad31_1586a0e02b73.slice/crio-1872b1268317d432cfb58a119c3f1c0ffd8839795928e3b7634f89c5cd823853 WatchSource:0}: Error finding container 1872b1268317d432cfb58a119c3f1c0ffd8839795928e3b7634f89c5cd823853: Status 404 returned error can't find the container with id 1872b1268317d432cfb58a119c3f1c0ffd8839795928e3b7634f89c5cd823853
Nov 03 21:09:41.451311 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:41.451275    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:41.555125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:41.555085    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" event={"ID":"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73","Type":"ContainerStarted","Data":"1872b1268317d432cfb58a119c3f1c0ffd8839795928e3b7634f89c5cd823853"}
Nov 03 21:09:42.589622 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:42.588756    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" event={"ID":"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73","Type":"ContainerStarted","Data":"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"}
Nov 03 21:09:42.589622 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:42.588880    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:42.713716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:42.713666    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" podStartSLOduration=2.812873597 podStartE2EDuration="3.713652671s" totalImagesPullingTime="900.779074ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:09:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:09:42.711552698 +0000 UTC m=+1797.162921876" watchObservedRunningTime="2026-11-03 21:09:42.713652671 +0000 UTC m=+1797.165021847"
Nov 03 21:09:43.532664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:43.532635    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:09:44.956375 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:44.956339    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:45.296037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:45.295955    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 21:09:45.414658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:45.414626    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:45.679629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:45.679493    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:45.968597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:45.967078    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:46.139750 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.139693    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:46.214388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.214361    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:09:46.218181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.218159    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:09:46.227360 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.227302    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:09:46.231963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.231946    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:09:46.826461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.826427    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:09:46.969586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:46.969382    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:48.601929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:48.601903    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:09:49.895320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:49.895227    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:50.304593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:50.302659    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing" containerID="cri-o://6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931" gracePeriod=25
Nov 03 21:09:50.626593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:50.626485    5915 scope.go:122] "RemoveContainer" containerID="ff970516946268cc9b756ff8e0e4301f27f23c55dfe9c7270dd25dcda5f31a16"
Nov 03 21:09:53.878599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:53.877744    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:09:54.534538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:54.534503    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:55.497743 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.497715    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 21:09:55.556203 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.556118    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:09:55.793277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.793182    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\" (UniqueName: \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\") pod \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") "
Nov 03 21:09:55.794758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.794728    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\") pod \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\" (UID: \"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d\") "
Nov 03 21:09:55.797322 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.796784    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume" pod "7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" (UID: "7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:09:55.887186 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.887129    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4" pod "7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" (UID: "7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d"). InnerVolumeSpecName "kube-api-access-m7cq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:55.891738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.891121    5915 generic.go:356] "Generic (PLEG): container finished" podID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerID="0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee" exitCode=0
Nov 03 21:09:55.891738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.891229    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" event={"ID":"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d","Type":"ContainerDied","Data":"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"}
Nov 03 21:09:55.891738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.891266    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb" event={"ID":"7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d","Type":"ContainerDied","Data":"adebfeb2e75da4a6501b4a2c6cc333bd3a955c429c918340fb9d498bbb3490cd"}
Nov 03 21:09:55.891738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.891290    5915 scope.go:122] "RemoveContainer" containerID="0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"
Nov 03 21:09:55.891738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.891413    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"
Nov 03 21:09:55.903649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.897460    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:55.903649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.897483    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7cq4\" (UniqueName: \"kubernetes.io/projected/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d-kube-api-access-m7cq4\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:55.989533 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.989513    5915 scope.go:122] "RemoveContainer" containerID="0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"
Nov 03 21:09:55.989948 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:09:55.989927    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee\": container with ID starting with 0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee not found: ID does not exist" containerID="0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"
Nov 03 21:09:55.990022 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:55.989957    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee"} err="failed to get container status \"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee\": rpc error: code = NotFound desc = could not find container \"0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee\": container with ID starting with 0a8580bf71d84512420fa6b63706fec061e1d7f0413ba5cd23d10d87bb80a2ee not found: ID does not exist"
Nov 03 21:09:56.094810 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:56.094728    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 21:09:56.152586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:56.134185    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-ca4642-554c6dd6f8-hkwhb"]
Nov 03 21:09:57.686061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:09:57.685984    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" path="/var/lib/kubelet/pods/7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d/volumes"
Nov 03 21:10:00.087827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:00.087789    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:00.673799 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:00.673762    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:01.785643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:01.785554    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:02.031993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:02.031332    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:02.319872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:02.206193    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:02.806718 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:02.806678    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:03.873989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:03.873956    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:10:04.050482 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.050448    5915 generic.go:356] "Generic (PLEG): container finished" podID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerID="02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43" exitCode=0
Nov 03 21:10:04.050685 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.050520    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerDied","Data":"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"}
Nov 03 21:10:04.051118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.051039    5915 scope.go:122] "RemoveContainer" containerID="02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"
Nov 03 21:10:04.052615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.052591    5915 generic.go:356] "Generic (PLEG): container finished" podID="93500355-da00-4f11-b925-11b8553bd9e6" containerID="db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e" exitCode=0
Nov 03 21:10:04.052730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.052672    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerDied","Data":"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"}
Nov 03 21:10:04.052855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.052828    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.052936 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.052918    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.052995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.052973    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.053079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.053064    5915 scope.go:122] "RemoveContainer" containerID="db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"
Nov 03 21:10:04.054627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.054600    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.054748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.054678    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.054748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.054728    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.056188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.056170    5915 generic.go:356] "Generic (PLEG): container finished" podID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerID="cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d" exitCode=0
Nov 03 21:10:04.056280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.056219    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerDied","Data":"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"}
Nov 03 21:10:04.056518 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.056497    5915 scope.go:122] "RemoveContainer" containerID="cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"
Nov 03 21:10:04.058307 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.058281    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.058390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.058354    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:04.058508 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:04.058397    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:05.082494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.082457    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerStarted","Data":"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"}
Nov 03 21:10:05.085865 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.085840    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerStarted","Data":"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"}
Nov 03 21:10:05.088319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.088300    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerStarted","Data":"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"}
Nov 03 21:10:05.274032 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.274000    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:10:05.274413 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.274386    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:05.274639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.274609    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:05.739064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.739029    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:10:05.739444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.739416    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:05.739629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:05.739604    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:06.063643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.062664    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:10:06.075158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.074750    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:10:06.075158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.074874    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 03 21:10:06.075158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.074917    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 03 21:10:06.273585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.269041    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:06.273585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.269102    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:06.680584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.680333    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:06.739583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.739167    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:06.739583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.739243    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:06.857582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:06.857364    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:07.076610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.074994    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 03 21:10:07.076610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.075058    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 03 21:10:07.198043 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:07.196895    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:10:07.279946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.278989    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:07.279946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.279046    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:07.746016 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.745963    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:07.746301 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.746269    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:07.867515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:07.867481    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:08.077151 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.077046    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 03 21:10:08.085141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.084665    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 03 21:10:08.271585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.271027    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:08.271585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.271089    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:08.739584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.736174    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:08.739584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:08.736235    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:09.085006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.084918    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 03 21:10:09.085006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.084985    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 03 21:10:09.269585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.268937    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:09.269585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.269002    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:09.547583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.544107    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:09.743620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.734682    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:09.743620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.734743    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:09.756585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:09.746691    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:10.127582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.125640    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Nov 03 21:10:10.127582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.125704    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Nov 03 21:10:10.140589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.137049    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:10.293671 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.293633    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 21:10:10.296438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.296401    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" containerID="cri-o://217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d" gracePeriod=30
Nov 03 21:10:10.297729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.297666    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused" start-of-body=
Nov 03 21:10:10.297854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.297731    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.130:15021/healthz/ready\": dial tcp 10.132.0.130:15021: connect: connection refused"
Nov 03 21:10:10.303741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.303717    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 21:10:10.304149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.304102    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" containerID="cri-o://48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6" gracePeriod=30
Nov 03 21:10:10.393044 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.350142    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:10.545587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.535055    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:10:10.545587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.535313    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" containerID="cri-o://1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb" gracePeriod=30
Nov 03 21:10:10.738588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.736002    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 03 21:10:10.738588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.736065    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 03 21:10:10.848491 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.842235    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:10.960400 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:10.960363    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 21:10:11.032114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.032019    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:10:11.032114 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.032081    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:10:11.049999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.049964    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\" (UniqueName: \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\") pod \"1ea5b727-d740-4455-8b7d-951460fe3443\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") "
Nov 03 21:10:11.050220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.050039    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\") pod \"1ea5b727-d740-4455-8b7d-951460fe3443\" (UID: \"1ea5b727-d740-4455-8b7d-951460fe3443\") "
Nov 03 21:10:11.050897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.050768    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume" pod "1ea5b727-d740-4455-8b7d-951460fe3443" (UID: "1ea5b727-d740-4455-8b7d-951460fe3443"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:11.053028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.052994    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm" pod "1ea5b727-d740-4455-8b7d-951460fe3443" (UID: "1ea5b727-d740-4455-8b7d-951460fe3443"). InnerVolumeSpecName "kube-api-access-8nmfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:11.151820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.151780    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:10:11.152488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.152459    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:10:11.158037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.153833    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nmfm\" (UniqueName: \"kubernetes.io/projected/1ea5b727-d740-4455-8b7d-951460fe3443-kube-api-access-8nmfm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:11.158037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.158002    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ea5b727-d740-4455-8b7d-951460fe3443-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:11.175529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.175497    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:10:11.274000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.273841    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:10:11.274275 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.274246    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:10:11.275252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.275231    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:10:11.383442 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.383085    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:11.535923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.535890    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:11.555679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.555645    5915 generic.go:356] "Generic (PLEG): container finished" podID="1ea5b727-d740-4455-8b7d-951460fe3443" containerID="6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931" exitCode=0
Nov 03 21:10:11.555869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.555724    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"
Nov 03 21:10:11.555869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.555739    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" event={"ID":"1ea5b727-d740-4455-8b7d-951460fe3443","Type":"ContainerDied","Data":"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"}
Nov 03 21:10:11.555869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.555802    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5" event={"ID":"1ea5b727-d740-4455-8b7d-951460fe3443","Type":"ContainerDied","Data":"6b4c351ace053dd9a28421485a9a018bc053d19a43b9d5e56131d3ba9fced1d8"}
Nov 03 21:10:11.555869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.555834    5915 scope.go:122] "RemoveContainer" containerID="6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"
Nov 03 21:10:11.569928 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.569902    5915 scope.go:122] "RemoveContainer" containerID="6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"
Nov 03 21:10:11.570299 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:11.570277    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931\": container with ID starting with 6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931 not found: ID does not exist" containerID="6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"
Nov 03 21:10:11.570425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.570307    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931"} err="failed to get container status \"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931\": rpc error: code = NotFound desc = could not find container \"6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931\": container with ID starting with 6d9fe1bb1cce2eaa38dc4474f3746c443335997a366b0a4ea7f1917952e0f931 not found: ID does not exist"
Nov 03 21:10:11.777580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.769935    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 21:10:11.777580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.770910    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-499187-c9f587f86-hg5c5"]
Nov 03 21:10:11.798611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.793685    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:10:11.798611 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:11.797724    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:10:12.794014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:12.793983    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:13.057741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.057655    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:13.684798 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.684769    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" path="/var/lib/kubelet/pods/1ea5b727-d740-4455-8b7d-951460fe3443/volumes"
Nov 03 21:10:13.748394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.748355    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:13.899205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.899173    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:13.907394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.907362    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:10:13.907598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:13.907427    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:10:14.649192 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:14.649151    5915 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:10:14.649360 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:14.649214    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:10:15.832083 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.832011    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 21:10:15.868214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.868181    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 21:10:15.908215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908185    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908228    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908268    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908298    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908385    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908449    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908538    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908604    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908634    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908675    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908672    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908679    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908766    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908796    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908814    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908800    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.908872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908862    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908911    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.908954    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909013    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909031    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909036    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909122    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\") pod \"a50bcc61-71b1-472c-933b-298d289cd55a\" (UID: \"a50bcc61-71b1-472c-933b-298d289cd55a\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909153    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\") pod \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\" (UID: \"7219df0e-e1e5-4e00-bf21-0f60ea3f865d\") "
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909410    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.909457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909432    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.909934 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909650    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:15.909934 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909737    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.909934 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909899    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.909934 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909918    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909935    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909949    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909965    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909980    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a50bcc61-71b1-472c-933b-298d289cd55a-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.909994    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.910009    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.910023    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.910149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.910037    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.911410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.911385    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:15.911410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.911395    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "kube-api-access-lxmzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.912247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.912223    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:15.912612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.912587    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.912703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.912621    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.912703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.912643    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "kube-api-access-k7sh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.912703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.912667    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token" pod "a50bcc61-71b1-472c-933b-298d289cd55a" (UID: "a50bcc61-71b1-472c-933b-298d289cd55a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.913049 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:15.913032    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy" pod "7219df0e-e1e5-4e00-bf21-0f60ea3f865d" (UID: "7219df0e-e1e5-4e00-bf21-0f60ea3f865d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.011838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011805    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a50bcc61-71b1-472c-933b-298d289cd55a-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.011838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011833    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a50bcc61-71b1-472c-933b-298d289cd55a-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.011838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011843    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.012120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011852    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.012120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011861    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.012120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011868    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.012120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011877    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7sh4\" (UniqueName: \"kubernetes.io/projected/a50bcc61-71b1-472c-933b-298d289cd55a-kube-api-access-k7sh4\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.012120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.011887    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxmzp\" (UniqueName: \"kubernetes.io/projected/7219df0e-e1e5-4e00-bf21-0f60ea3f865d-kube-api-access-lxmzp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.283147 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.283121    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:10:16.418381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418285    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418353    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418424    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418458    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418525    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418650 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418605    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418666    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418763    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.418882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418805    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\") pod \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\" (UID: \"01376f4d-aefb-44fc-bd45-3af8a8c295e0\") "
Nov 03 21:10:16.419035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.418876    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.419185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419159    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:16.419262 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419185    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.419311 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419278    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.419424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419403    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.419605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419579    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.419908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419609    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.419908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419624    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.419908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419641    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.419908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.419655    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.421850 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.421823    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:16.421950 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.421850    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "kube-api-access-xnz8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:16.421950 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.421883    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:16.423762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.423739    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy" pod "01376f4d-aefb-44fc-bd45-3af8a8c295e0" (UID: "01376f4d-aefb-44fc-bd45-3af8a8c295e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:16.521222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.521190    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.521222 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.521220    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.521426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.521230    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01376f4d-aefb-44fc-bd45-3af8a8c295e0-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.521426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.521250    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnz8t\" (UniqueName: \"kubernetes.io/projected/01376f4d-aefb-44fc-bd45-3af8a8c295e0-kube-api-access-xnz8t\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:16.598705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.598669    5915 generic.go:356] "Generic (PLEG): container finished" podID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerID="48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6" exitCode=0
Nov 03 21:10:16.598838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.598719    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"
Nov 03 21:10:16.598838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.598752    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" event={"ID":"7219df0e-e1e5-4e00-bf21-0f60ea3f865d","Type":"ContainerDied","Data":"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"}
Nov 03 21:10:16.598838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.598813    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk" event={"ID":"7219df0e-e1e5-4e00-bf21-0f60ea3f865d","Type":"ContainerDied","Data":"3946268fb5bd7bda58ccdc6b2269efad440eeba2976a84a2f231d96f2e47c5a3"}
Nov 03 21:10:16.598838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.598835    5915 scope.go:122] "RemoveContainer" containerID="48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"
Nov 03 21:10:16.600389 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.600360    5915 generic.go:356] "Generic (PLEG): container finished" podID="a50bcc61-71b1-472c-933b-298d289cd55a" containerID="217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d" exitCode=0
Nov 03 21:10:16.600526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.600404    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" event={"ID":"a50bcc61-71b1-472c-933b-298d289cd55a","Type":"ContainerDied","Data":"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"}
Nov 03 21:10:16.600526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.600459    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d" event={"ID":"a50bcc61-71b1-472c-933b-298d289cd55a","Type":"ContainerDied","Data":"e421978dae3c464b83a2fc757ae3f0dfb661f9e9d2af6d9dac826734afe1d1f3"}
Nov 03 21:10:16.600526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.600415    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"
Nov 03 21:10:16.602150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.602126    5915 generic.go:356] "Generic (PLEG): container finished" podID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerID="1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb" exitCode=0
Nov 03 21:10:16.602273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.602157    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" event={"ID":"01376f4d-aefb-44fc-bd45-3af8a8c295e0","Type":"ContainerDied","Data":"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"}
Nov 03 21:10:16.602273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.602174    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"
Nov 03 21:10:16.602273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.602197    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77" event={"ID":"01376f4d-aefb-44fc-bd45-3af8a8c295e0","Type":"ContainerDied","Data":"e4056811ab91188089a256f6ff6639cb411528dc4df35ad7392a5c5162c2be8e"}
Nov 03 21:10:16.614236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.614061    5915 scope.go:122] "RemoveContainer" containerID="48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"
Nov 03 21:10:16.614520 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:16.614451    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6\": container with ID starting with 48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6 not found: ID does not exist" containerID="48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"
Nov 03 21:10:16.614520 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.614478    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6"} err="failed to get container status \"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6\": rpc error: code = NotFound desc = could not find container \"48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6\": container with ID starting with 48f44aac8e5c56c7e40e4c5bd8048f70324b9560fe0948e3d4111abdaccfe0a6 not found: ID does not exist"
Nov 03 21:10:16.614520 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.614500    5915 scope.go:122] "RemoveContainer" containerID="217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"
Nov 03 21:10:16.624714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.624696    5915 scope.go:122] "RemoveContainer" containerID="217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"
Nov 03 21:10:16.624987 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:16.624966    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d\": container with ID starting with 217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d not found: ID does not exist" containerID="217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"
Nov 03 21:10:16.625086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.624993    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d"} err="failed to get container status \"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d\": rpc error: code = NotFound desc = could not find container \"217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d\": container with ID starting with 217e62acb2dd7dc5f732598f0997e9e83d3a6b6d738d0a0e3eb690d56e60d54d not found: ID does not exist"
Nov 03 21:10:16.625086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.625010    5915 scope.go:122] "RemoveContainer" containerID="1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"
Nov 03 21:10:16.634549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.634531    5915 scope.go:122] "RemoveContainer" containerID="1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"
Nov 03 21:10:16.634904 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:16.634885    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb\": container with ID starting with 1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb not found: ID does not exist" containerID="1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"
Nov 03 21:10:16.634980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.634909    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb"} err="failed to get container status \"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb\": rpc error: code = NotFound desc = could not find container \"1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb\": container with ID starting with 1ce77b7b86c97d08f0c5285d6dbe66312f8c8369f67d449751459e8fe09c59bb not found: ID does not exist"
Nov 03 21:10:16.764967 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.764933    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:10:16.768695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.768672    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-sbt77"]
Nov 03 21:10:16.878193 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.878166    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 21:10:16.884573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.884530    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-7cspk"]
Nov 03 21:10:16.993632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.993597    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 21:10:16.999524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:16.999496    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-svs4d"]
Nov 03 21:10:17.242012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.241983    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:17.421080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.421052    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:17.683205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.683129    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" path="/var/lib/kubelet/pods/01376f4d-aefb-44fc-bd45-3af8a8c295e0/volumes"
Nov 03 21:10:17.683659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.683644    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" path="/var/lib/kubelet/pods/7219df0e-e1e5-4e00-bf21-0f60ea3f865d/volumes"
Nov 03 21:10:17.684096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.684083    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" path="/var/lib/kubelet/pods/a50bcc61-71b1-472c-933b-298d289cd55a/volumes"
Nov 03 21:10:17.704355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:17.704304    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:19.074789 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:19.074744    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:19.559272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:19.559236    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:21.603828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:21.603797    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:22.116110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:22.116071    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:22.799579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:22.793514    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:23.211176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:23.211142    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:24.665578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:24.664955    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:24.998387 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:24.961149    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:25.141948 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:25.141890    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:27.585706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:27.585655    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:27.775710 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:27.775676    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:27.962353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:27.962319    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:30.653668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:30.653628    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:31.063509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:31.063369    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:31.249670 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:31.249353    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:32.335763 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:32.335725    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:34.831098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.831062    5915 generic.go:356] "Generic (PLEG): container finished" podID="fb672755-dd4e-430d-af31-4da8f757e02c" containerID="bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b" exitCode=0
Nov 03 21:10:34.831696 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.831135    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerDied","Data":"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"}
Nov 03 21:10:34.831696 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.831494    5915 scope.go:122] "RemoveContainer" containerID="bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"
Nov 03 21:10:34.832948 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.832923    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:34.833012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.832990    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:34.833050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:34.833023    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:10:35.840897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:35.840868    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerStarted","Data":"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"}
Nov 03 21:10:35.972270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:35.972214    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:36.489803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:36.489770    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:10:36.489999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:36.489886    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 03 21:10:36.489999 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:36.489936    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 03 21:10:37.028884 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:10:37.028837    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:10:37.490328 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:37.490285    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 03 21:10:37.490575 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:37.490379    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 03 21:10:38.489910 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:38.489874    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 03 21:10:38.490385 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:38.489934    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 03 21:10:39.490502 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:39.490465    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 03 21:10:39.490932 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:39.490522    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 03 21:10:40.489704 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:40.489670    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 03 21:10:40.489902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:40.489725    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 03 21:10:41.146890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.146858    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:41.309929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.309888    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:41.562970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.562943    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:10:41.564251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.564228    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:10:41.564899 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.564881    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:10:41.825771 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:41.825687    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:45.904990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:45.904955    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:46.836778 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:46.836739    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:47.256990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:47.256955    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:47.459918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:47.459892    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:47.620437 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:47.620365    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:47.775182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:47.775151    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:47.930370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:47.930267    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:48.126610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:48.119597    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:48.447268 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:48.447226    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:52.648126 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:52.648090    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:52.866659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:52.866623    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:53.056226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:53.056185    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:54.976309 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:54.976273    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:10:55.032969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:55.032939    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:55.132432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:55.132399    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:10:55.188820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:55.188795    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:55.364809 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:55.364732    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:55.534470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:55.534436    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:56.198617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.198585    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:56.305583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.305539    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:10:56.306052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.306014    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" containerID="cri-o://edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8" gracePeriod=30
Nov 03 21:10:56.374632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.374601    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:56.629426 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.629369    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:56.630294 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.630270    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 21:10:56.631143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:56.631106    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy" containerID="cri-o://a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc" gracePeriod=30
Nov 03 21:10:58.328885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:58.328849    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:59.014808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:59.014755    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:59.243047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:59.243010    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:10:59.977093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:10:59.977049    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" podUID="5d7c2529-c044-4830-a99e-ebb22e77acb3" containerName="payload-processing" containerID="cri-o://5e52fb4b50ec9139e314666a3c472a635f37f704fd5f31fe418cfd2307f2759a" gracePeriod=25
Nov 03 21:11:00.133349 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.133301    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" podUID="5f7b2612-576e-43ed-b659-13220ac034c1" containerName="payload-pre-processing" containerID="cri-o://25b76d38db47aebffbe698c05d98f176d1737d8fa20d9b162fa37a15674f5a40" gracePeriod=25
Nov 03 21:11:00.269300 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.269262    5915 generic.go:356] "Generic (PLEG): container finished" podID="5f7b2612-576e-43ed-b659-13220ac034c1" containerID="25b76d38db47aebffbe698c05d98f176d1737d8fa20d9b162fa37a15674f5a40" exitCode=0
Nov 03 21:11:00.269461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.269335    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" event={"ID":"5f7b2612-576e-43ed-b659-13220ac034c1","Type":"ContainerDied","Data":"25b76d38db47aebffbe698c05d98f176d1737d8fa20d9b162fa37a15674f5a40"}
Nov 03 21:11:00.271749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.271722    5915 generic.go:356] "Generic (PLEG): container finished" podID="5d7c2529-c044-4830-a99e-ebb22e77acb3" containerID="5e52fb4b50ec9139e314666a3c472a635f37f704fd5f31fe418cfd2307f2759a" exitCode=0
Nov 03 21:11:00.271877 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.271792    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" event={"ID":"5d7c2529-c044-4830-a99e-ebb22e77acb3","Type":"ContainerDied","Data":"5e52fb4b50ec9139e314666a3c472a635f37f704fd5f31fe418cfd2307f2759a"}
Nov 03 21:11:00.271877 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.271835    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb" event={"ID":"5d7c2529-c044-4830-a99e-ebb22e77acb3","Type":"ContainerDied","Data":"e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba"}
Nov 03 21:11:00.271877 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.271857    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4a17ab33e142cf2138e253ef909d2a208cec7f8f1309cc758acb361d1cf49ba"
Nov 03 21:11:00.273498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.273478    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:11:00.375588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.375540    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\" (UniqueName: \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\") pod \"5d7c2529-c044-4830-a99e-ebb22e77acb3\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") "
Nov 03 21:11:00.375753 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.375686    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\") pod \"5d7c2529-c044-4830-a99e-ebb22e77acb3\" (UID: \"5d7c2529-c044-4830-a99e-ebb22e77acb3\") "
Nov 03 21:11:00.376263 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.376231    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume" pod "5d7c2529-c044-4830-a99e-ebb22e77acb3" (UID: "5d7c2529-c044-4830-a99e-ebb22e77acb3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:00.376498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.376472    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c2529-c044-4830-a99e-ebb22e77acb3-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:00.378362 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.378340    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm" pod "5d7c2529-c044-4830-a99e-ebb22e77acb3" (UID: "5d7c2529-c044-4830-a99e-ebb22e77acb3"). InnerVolumeSpecName "kube-api-access-xlhvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:00.389828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.389809    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:11:00.478083 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.478054    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlhvm\" (UniqueName: \"kubernetes.io/projected/5d7c2529-c044-4830-a99e-ebb22e77acb3-kube-api-access-xlhvm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:00.579429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.579359    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\" (UniqueName: \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\") pod \"5f7b2612-576e-43ed-b659-13220ac034c1\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") "
Nov 03 21:11:00.579429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.579413    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\") pod \"5f7b2612-576e-43ed-b659-13220ac034c1\" (UID: \"5f7b2612-576e-43ed-b659-13220ac034c1\") "
Nov 03 21:11:00.579920 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.579880    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume" pod "5f7b2612-576e-43ed-b659-13220ac034c1" (UID: "5f7b2612-576e-43ed-b659-13220ac034c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:00.581619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.581595    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2" pod "5f7b2612-576e-43ed-b659-13220ac034c1" (UID: "5f7b2612-576e-43ed-b659-13220ac034c1"). InnerVolumeSpecName "kube-api-access-z7bh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:00.680798 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.680755    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7bh2\" (UniqueName: \"kubernetes.io/projected/5f7b2612-576e-43ed-b659-13220ac034c1-kube-api-access-z7bh2\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:00.680798 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.680793    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f7b2612-576e-43ed-b659-13220ac034c1-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:00.751185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.751149    5915 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:11:00.751326 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:00.751201    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:11:01.281208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.281134    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"
Nov 03 21:11:01.281208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.281175    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp" event={"ID":"5f7b2612-576e-43ed-b659-13220ac034c1","Type":"ContainerDied","Data":"0c02c6a63959ad836875b38306095c449e506511f1bd88fe216ffae52ed0815c"}
Nov 03 21:11:01.281715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.281246    5915 scope.go:122] "RemoveContainer" containerID="25b76d38db47aebffbe698c05d98f176d1737d8fa20d9b162fa37a15674f5a40"
Nov 03 21:11:01.281715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.281352    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"
Nov 03 21:11:01.453319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.453282    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:11:01.499679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.499084    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-206508-7bb98c9499-7khxp"]
Nov 03 21:11:01.602113 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.602090    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:11:01.607002 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.606979    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:11:01.633150 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.633129    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-206508-694db7ccdd-hgrfb"]
Nov 03 21:11:01.683361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.683335    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d7c2529-c044-4830-a99e-ebb22e77acb3" path="/var/lib/kubelet/pods/5d7c2529-c044-4830-a99e-ebb22e77acb3/volumes"
Nov 03 21:11:01.683834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.683820    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f7b2612-576e-43ed-b659-13220ac034c1" path="/var/lib/kubelet/pods/5f7b2612-576e-43ed-b659-13220ac034c1/volumes"
Nov 03 21:11:01.694741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694721    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.694830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694756    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.694830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694781    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.694913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694859    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.694959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694937    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.695000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.694960    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:01.695047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695031    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.695081 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695039    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:01.695081 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695073    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.695172 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695152    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:01.695213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695173    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.695213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695191    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\") pod \"cbb95f81-3b62-456e-9294-2505824fbc81\" (UID: \"cbb95f81-3b62-456e-9294-2505824fbc81\") "
Nov 03 21:11:01.695518 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695462    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:01.695629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695571    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:01.695893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695736    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbb95f81-3b62-456e-9294-2505824fbc81-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.695893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695761    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.695893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695773    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.695893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695783    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.695893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.695792    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.697874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.697845    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:01.697874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.697860    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:01.697874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.697858    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:01.698047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.697862    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6" pod "cbb95f81-3b62-456e-9294-2505824fbc81" (UID: "cbb95f81-3b62-456e-9294-2505824fbc81"). InnerVolumeSpecName "kube-api-access-pf6c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:01.797434 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.797400    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbb95f81-3b62-456e-9294-2505824fbc81-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.797434 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.797434    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf6c6\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-kube-api-access-pf6c6\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.797659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.797447    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbb95f81-3b62-456e-9294-2505824fbc81-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.797659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.797459    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbb95f81-3b62-456e-9294-2505824fbc81-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:01.917166 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:01.917138    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 21:11:02.102227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102150    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102197    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102239    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102399    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102518    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102593    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.102623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102604    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102676    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.102794 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102756    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102786    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.102907 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102882    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.102957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102942    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\") pod \"77f6e328-9dc0-4910-8943-a50541d7876f\" (UID: \"77f6e328-9dc0-4910-8943-a50541d7876f\") "
Nov 03 21:11:02.103014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.102984    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.103429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.103347    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:02.104463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.104437    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f6e328-9dc0-4910-8943-a50541d7876f-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.104664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.104637    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.104664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.104663    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.104853 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.104675    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.104853 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.104687    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.105108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.105083    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:02.105214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.105105    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:02.105214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.105205    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "kube-api-access-fgbmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:02.105378 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.105361    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy" pod "77f6e328-9dc0-4910-8943-a50541d7876f" (UID: "77f6e328-9dc0-4910-8943-a50541d7876f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.206224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.206196    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f6e328-9dc0-4910-8943-a50541d7876f-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.206224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.206220    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.206224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.206229    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgbmg\" (UniqueName: \"kubernetes.io/projected/77f6e328-9dc0-4910-8943-a50541d7876f-kube-api-access-fgbmg\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.206438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.206241    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f6e328-9dc0-4910-8943-a50541d7876f-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.291106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.291071    5915 generic.go:356] "Generic (PLEG): container finished" podID="cbb95f81-3b62-456e-9294-2505824fbc81" containerID="edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8" exitCode=0
Nov 03 21:11:02.291531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.291116    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"
Nov 03 21:11:02.291531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.291123    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" event={"ID":"cbb95f81-3b62-456e-9294-2505824fbc81","Type":"ContainerDied","Data":"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"}
Nov 03 21:11:02.291531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.291186    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft" event={"ID":"cbb95f81-3b62-456e-9294-2505824fbc81","Type":"ContainerDied","Data":"1295d877e136850a11eeed25c3bcc3cb6980c72355356782dbe908d54eb7c634"}
Nov 03 21:11:02.291531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.291219    5915 scope.go:122] "RemoveContainer" containerID="edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"
Nov 03 21:11:02.292677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.292657    5915 generic.go:356] "Generic (PLEG): container finished" podID="77f6e328-9dc0-4910-8943-a50541d7876f" containerID="a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc" exitCode=0
Nov 03 21:11:02.292765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.292702    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"
Nov 03 21:11:02.292765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.292736    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" event={"ID":"77f6e328-9dc0-4910-8943-a50541d7876f","Type":"ContainerDied","Data":"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"}
Nov 03 21:11:02.292857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.292784    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh" event={"ID":"77f6e328-9dc0-4910-8943-a50541d7876f","Type":"ContainerDied","Data":"74d51e7cd890b84f7df5852b3e91ad1ec9f192dc6988dafe34575279e565b499"}
Nov 03 21:11:02.303620 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.303540    5915 scope.go:122] "RemoveContainer" containerID="edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"
Nov 03 21:11:02.303844 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:02.303827    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8\": container with ID starting with edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8 not found: ID does not exist" containerID="edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"
Nov 03 21:11:02.303891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.303853    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8"} err="failed to get container status \"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8\": rpc error: code = NotFound desc = could not find container \"edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8\": container with ID starting with edfe01ccc25453430bf22d018c3685490714d944f99d3d3cb8f8d6317f4e61f8 not found: ID does not exist"
Nov 03 21:11:02.303891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.303873    5915 scope.go:122] "RemoveContainer" containerID="a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"
Nov 03 21:11:02.315794 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.315713    5915 scope.go:122] "RemoveContainer" containerID="a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"
Nov 03 21:11:02.316191 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:02.316122    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc\": container with ID starting with a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc not found: ID does not exist" containerID="a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"
Nov 03 21:11:02.316191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.316156    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc"} err="failed to get container status \"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc\": rpc error: code = NotFound desc = could not find container \"a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc\": container with ID starting with a93f1fc264a09cb5c8444f332ca83ea1820b48e0d2a23fafee84a1b4e23d56dc not found: ID does not exist"
Nov 03 21:11:02.457194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.457166    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:11:02.506228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.506190    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-g9pft"]
Nov 03 21:11:02.618440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.618409    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 21:11:02.621787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:02.621768    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-98smh"]
Nov 03 21:11:03.334363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:03.334333    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:03.683106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:03.683028    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" path="/var/lib/kubelet/pods/77f6e328-9dc0-4910-8943-a50541d7876f/volumes"
Nov 03 21:11:03.683572 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:03.683540    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" path="/var/lib/kubelet/pods/cbb95f81-3b62-456e-9294-2505824fbc81/volumes"
Nov 03 21:11:07.006004 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:07.005969    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:11:08.133745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:08.133703    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:12.620765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.620724    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:11:12.621521 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621497    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621522    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621546    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621570    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621585    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5f7b2612-576e-43ed-b659-13220ac034c1" containerName="payload-pre-processing"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621593    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f7b2612-576e-43ed-b659-13220ac034c1" containerName="payload-pre-processing"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621613    5915 cpu_manager.go:414] "containerMap: removing container" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621620    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621656    5915 cpu_manager.go:414] "containerMap: removing container" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621662    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy"
Nov 03 21:11:12.621687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621691    5915 cpu_manager.go:414] "containerMap: removing container" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621698    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621719    5915 cpu_manager.go:414] "containerMap: removing container" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621726    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621742    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5d7c2529-c044-4830-a99e-ebb22e77acb3" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621749    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d7c2529-c044-4830-a99e-ebb22e77acb3" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621762    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621768    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621936    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7219df0e-e1e5-4e00-bf21-0f60ea3f865d" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621948    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f7b2612-576e-43ed-b659-13220ac034c1" containerName="payload-pre-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621964    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbb95f81-3b62-456e-9294-2505824fbc81" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621980    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="77f6e328-9dc0-4910-8943-a50541d7876f" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.621993    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="01376f4d-aefb-44fc-bd45-3af8a8c295e0" containerName="istio-proxy"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.622004    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ea5b727-d740-4455-8b7d-951460fe3443" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.622016    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d7c2529-c044-4830-a99e-ebb22e77acb3" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.622026    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cae2c3f-6804-4710-bfd6-6a22bc7fbb5d" containerName="payload-processing"
Nov 03 21:11:12.622194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.622037    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a50bcc61-71b1-472c-933b-298d289cd55a" containerName="istio-proxy"
Nov 03 21:11:12.625917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.625897    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.678355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.678328    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-32c41e-serving-cert\""
Nov 03 21:11:12.731339 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.731308    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:11:12.756855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.756813    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.757069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.756871    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br89l\" (UniqueName: \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.757069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.757038    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.757235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.757127    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.811789 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.811760    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.862718    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.862779    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br89l\" (UniqueName: \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.862955    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.863049    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:12.863373    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-serving-cert: secret "maas-api-e2e-shared-b-w5-32c41e-serving-cert" not found
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:12.863497    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls podName:d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e nodeName:}" failed. No retries permitted until 2026-11-03 21:11:13.363429905 +0000 UTC m=+1887.814799064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls") pod "maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" (UID: "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e") : secret "maas-api-e2e-shared-b-w5-32c41e-serving-cert" not found
Nov 03 21:11:12.876639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.870509    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.925581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.924768    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.932725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.932691    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br89l\" (UniqueName: \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:12.982358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:12.982324    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:13.130767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:13.130680    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:13.370255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:13.370217    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:13.373020 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:13.372988    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:13.540760 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:13.540725    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:13.930068 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:13.930042    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:11:13.930730 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:11:13.930702    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd72c7317_46e2_4ba5_8bcb_e03b3e8ec57e.slice/crio-33a57c043ecd2ec02264a9487bf5b6cb8a2e729cc1ac5485713d34498d0a3827 WatchSource:0}: Error finding container 33a57c043ecd2ec02264a9487bf5b6cb8a2e729cc1ac5485713d34498d0a3827: Status 404 returned error can't find the container with id 33a57c043ecd2ec02264a9487bf5b6cb8a2e729cc1ac5485713d34498d0a3827
Nov 03 21:11:14.386960 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:14.386925    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" event={"ID":"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e","Type":"ContainerStarted","Data":"33a57c043ecd2ec02264a9487bf5b6cb8a2e729cc1ac5485713d34498d0a3827"}
Nov 03 21:11:15.395637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:15.395595    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" event={"ID":"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e","Type":"ContainerStarted","Data":"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"}
Nov 03 21:11:15.396055 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:15.395684    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:15.515433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:15.515376    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" podStartSLOduration=2.722740448 podStartE2EDuration="3.515362451s" totalImagesPullingTime="792.622003ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:11:15.512502183 +0000 UTC m=+1889.963871360" watchObservedRunningTime="2026-11-03 21:11:15.515362451 +0000 UTC m=+1889.966731628"
Nov 03 21:11:17.155785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.155752    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:11:17.156244 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.156007    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" podUID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" containerName="maas-api" containerID="cri-o://454b08b42d40c620095011e4b977948ce508782aca2f7ba402aeccf9a11f59f9" gracePeriod=30
Nov 03 21:11:17.260981 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.260948    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:17.453861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.453829    5915 generic.go:356] "Generic (PLEG): container finished" podID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" containerID="454b08b42d40c620095011e4b977948ce508782aca2f7ba402aeccf9a11f59f9" exitCode=0
Nov 03 21:11:17.454052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.454019    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" event={"ID":"0c0c6ad9-eabd-4b30-9424-1e841aba47ff","Type":"ContainerDied","Data":"454b08b42d40c620095011e4b977948ce508782aca2f7ba402aeccf9a11f59f9"}
Nov 03 21:11:17.496549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.496527    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:11:17.672394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.672363    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\") pod \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") "
Nov 03 21:11:17.672618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.672514    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") pod \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") "
Nov 03 21:11:17.672618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.672543    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\" (UniqueName: \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\") pod \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") "
Nov 03 21:11:17.672772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.672634    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\") pod \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\" (UID: \"0c0c6ad9-eabd-4b30-9424-1e841aba47ff\") "
Nov 03 21:11:17.673321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.673297    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca" pod "0c0c6ad9-eabd-4b30-9424-1e841aba47ff" (UID: "0c0c6ad9-eabd-4b30-9424-1e841aba47ff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:17.675352 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.675328    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf" pod "0c0c6ad9-eabd-4b30-9424-1e841aba47ff" (UID: "0c0c6ad9-eabd-4b30-9424-1e841aba47ff"). InnerVolumeSpecName "kube-api-access-fb6kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:17.675468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.675437    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs" pod "0c0c6ad9-eabd-4b30-9424-1e841aba47ff" (UID: "0c0c6ad9-eabd-4b30-9424-1e841aba47ff"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:11:17.677051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.677028    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls" pod "0c0c6ad9-eabd-4b30-9424-1e841aba47ff" (UID: "0c0c6ad9-eabd-4b30-9424-1e841aba47ff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:11:17.775603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.775274    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:17.775603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.775319    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:17.775603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.775336    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:17.775603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.775351    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fb6kf\" (UniqueName: \"kubernetes.io/projected/0c0c6ad9-eabd-4b30-9424-1e841aba47ff-kube-api-access-fb6kf\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:17.850259 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:17.850223    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:18.436865 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.436831    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:18.462628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.462605    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"
Nov 03 21:11:18.462628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.462613    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl" event={"ID":"0c0c6ad9-eabd-4b30-9424-1e841aba47ff","Type":"ContainerDied","Data":"fcbece92cc0ab87c34b2f67240f452a011ec591bf4f5f839aa8a1cba2acf8b46"}
Nov 03 21:11:18.462881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.462666    5915 scope.go:122] "RemoveContainer" containerID="454b08b42d40c620095011e4b977948ce508782aca2f7ba402aeccf9a11f59f9"
Nov 03 21:11:18.631619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.631584    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:11:18.680824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.680793    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-206508-57f4f79fbb-7klsl"]
Nov 03 21:11:18.811627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:18.811587    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:19.063509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:19.063437    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:19.682460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:19.682431    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" path="/var/lib/kubelet/pods/0c0c6ad9-eabd-4b30-9424-1e841aba47ff/volumes"
Nov 03 21:11:19.756639 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:19.756604    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:20.056402 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:20.056357    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:20.344460 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:20.344377    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:20.512041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:20.512012    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:21.017766 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:21.017733    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:21.422000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:21.421868    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:11:25.511154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.511071    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:11:25.512232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.511702    5915 cpu_manager.go:414] "containerMap: removing container" podUID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" containerName="maas-api"
Nov 03 21:11:25.512232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.511715    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" containerName="maas-api"
Nov 03 21:11:25.512232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.511839    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0c6ad9-eabd-4b30-9424-1e841aba47ff" containerName="maas-api"
Nov 03 21:11:25.516839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.516820    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.570651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.570630    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-1e5494-serving-cert\""
Nov 03 21:11:25.621099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.621068    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.621249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.621190    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dms9\" (UniqueName: \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.621362 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.621347    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.621401 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.621385    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.623446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.623428    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:11:25.722897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.722869    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.723054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.722927    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dms9\" (UniqueName: \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.723054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.723032    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.723143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.723061    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.723190 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:25.723164    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-serving-cert: secret "maas-api-e2e-shared-b-w6-1e5494-serving-cert" not found
Nov 03 21:11:25.723243 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:25.723232    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls podName:018ac50c-c050-4ee3-b2d8-38d7b0dd89b0 nodeName:}" failed. No retries permitted until 2026-11-03 21:11:26.22321785 +0000 UTC m=+1900.674587004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls") pod "maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" (UID: "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0") : secret "maas-api-e2e-shared-b-w6-1e5494-serving-cert" not found
Nov 03 21:11:25.723644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.723627    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.725545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.725525    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.805060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.804989    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dms9\" (UniqueName: \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:25.929034 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:25.929000    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:26.111810 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:26.111724    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:26.238612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:26.232668    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:26.278580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:26.270184    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:26.432388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:26.432306    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:26.906414 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:26.906385    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:11:26.908824 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:11:26.908798    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018ac50c_c050_4ee3_b2d8_38d7b0dd89b0.slice/crio-8e4b83ae281010c1fb249e513d3b43685e1f010a5e143678bdbc91ced67ad187 WatchSource:0}: Error finding container 8e4b83ae281010c1fb249e513d3b43685e1f010a5e143678bdbc91ced67ad187: Status 404 returned error can't find the container with id 8e4b83ae281010c1fb249e513d3b43685e1f010a5e143678bdbc91ced67ad187
Nov 03 21:11:27.281884 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.281848    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:27.431300 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.431271    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:27.549895 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.549807    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" event={"ID":"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0","Type":"ContainerStarted","Data":"8e4b83ae281010c1fb249e513d3b43685e1f010a5e143678bdbc91ced67ad187"}
Nov 03 21:11:27.630164 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.630133    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:11:27.630366 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.630346    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy" containerID="cri-o://cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6" gracePeriod=30
Nov 03 21:11:27.660855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:27.660830    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:28.560572 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:28.560525    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" event={"ID":"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0","Type":"ContainerStarted","Data":"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"}
Nov 03 21:11:28.561028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:28.560638    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:28.685983 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:28.685935    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" podStartSLOduration=2.755464882 podStartE2EDuration="3.685920172s" totalImagesPullingTime="930.45529ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:11:28.684356846 +0000 UTC m=+1903.135726036" watchObservedRunningTime="2026-11-03 21:11:28.685920172 +0000 UTC m=+1903.137289348"
Nov 03 21:11:28.776952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:28.776917    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:28.927239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:28.927158    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:29.086592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.084147    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 21:11:29.086592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.084376    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy" containerID="cri-o://d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273" gracePeriod=30
Nov 03 21:11:29.113060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.111912    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:29.118384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.116997    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 21:11:29.118384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.117227    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy" containerID="cri-o://5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01" gracePeriod=30
Nov 03 21:11:29.129141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.129114    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:11:29.129316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.129299    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" containerID="cri-o://a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00" gracePeriod=30
Nov 03 21:11:29.280836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:29.279923    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:30.069854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:30.069816    5915 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:11:30.070283 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:30.069881    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:11:31.523969 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:31.523932    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:32.916984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:32.916956    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:11:33.040072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040043    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040092    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040142    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040207    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040258    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040352    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040341    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:33.040415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040393    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040464    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040528    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\") pod \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\" (UID: \"4adbdd4a-ba51-4c24-a2f9-421056d099ca\") "
Nov 03 21:11:33.040683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040604    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:33.040683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040550    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:33.040683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040641    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:33.040921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.040679    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:33.041094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.041076    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.041226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.041098    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.041226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.041112    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.041226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.041124    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.041226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.041137    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.042738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.042716    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:33.042994 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.042970    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "kube-api-access-lnlqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:33.043202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.043184    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:33.043202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.043190    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy" pod "4adbdd4a-ba51-4c24-a2f9-421056d099ca" (UID: "4adbdd4a-ba51-4c24-a2f9-421056d099ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:33.142403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.142369    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.142403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.142399    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.142403 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.142410    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnlqb\" (UniqueName: \"kubernetes.io/projected/4adbdd4a-ba51-4c24-a2f9-421056d099ca-kube-api-access-lnlqb\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.142660 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.142419    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4adbdd4a-ba51-4c24-a2f9-421056d099ca-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:33.610227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.610196    5915 generic.go:356] "Generic (PLEG): container finished" podID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerID="cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6" exitCode=0
Nov 03 21:11:33.610386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.610250    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"
Nov 03 21:11:33.610386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.610288    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerDied","Data":"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"}
Nov 03 21:11:33.610386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.610348    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2" event={"ID":"4adbdd4a-ba51-4c24-a2f9-421056d099ca","Type":"ContainerDied","Data":"c923418a10558056c19a262fadc18d6b235ad1bfe58ade142ec1a6339ed739a9"}
Nov 03 21:11:33.610386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.610379    5915 scope.go:122] "RemoveContainer" containerID="cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"
Nov 03 21:11:33.622753 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.622734    5915 scope.go:122] "RemoveContainer" containerID="cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"
Nov 03 21:11:33.632755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.632713    5915 scope.go:122] "RemoveContainer" containerID="cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"
Nov 03 21:11:33.633021 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:33.633000    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6\": container with ID starting with cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6 not found: ID does not exist" containerID="cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"
Nov 03 21:11:33.633107 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.633026    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6"} err="failed to get container status \"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6\": rpc error: code = NotFound desc = could not find container \"cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6\": container with ID starting with cb5254c9e25bcf31e537f880a1a27b19e90c714a6984f1301529feda6a7901d6 not found: ID does not exist"
Nov 03 21:11:33.633107 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.633044    5915 scope.go:122] "RemoveContainer" containerID="cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"
Nov 03 21:11:33.633259 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:33.633243    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b\": container with ID starting with cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b not found: ID does not exist" containerID="cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"
Nov 03 21:11:33.633300 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.633262    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b"} err="failed to get container status \"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b\": rpc error: code = NotFound desc = could not find container \"cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b\": container with ID starting with cafb0aa27044cfbd2c3f9c03d4235c77f3078e2bd272b01f89141e59505dfa3b not found: ID does not exist"
Nov 03 21:11:33.825706 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.825678    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:11:33.875075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:33.875005    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-206508-openshift-default-775644f64c-b4kf2"]
Nov 03 21:11:34.529742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.529709    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 21:11:34.560446 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.560425    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:11:34.574213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.574190    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:11:34.584484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.584465    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 21:11:34.618583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.618529    5915 generic.go:356] "Generic (PLEG): container finished" podID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerID="5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01" exitCode=0
Nov 03 21:11:34.618700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.618621    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"
Nov 03 21:11:34.618700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.618618    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" event={"ID":"9e8190bf-c316-4f3a-bafe-517a8b3254fb","Type":"ContainerDied","Data":"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"}
Nov 03 21:11:34.618700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.618679    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g" event={"ID":"9e8190bf-c316-4f3a-bafe-517a8b3254fb","Type":"ContainerDied","Data":"174c7a37421d7dc62648732241b6136ee4750db51ad2f2ec4a99b575c0ae09ed"}
Nov 03 21:11:34.618885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.618709    5915 scope.go:122] "RemoveContainer" containerID="5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"
Nov 03 21:11:34.621104 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.621085    5915 generic.go:356] "Generic (PLEG): container finished" podID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerID="d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273" exitCode=0
Nov 03 21:11:34.621213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.621121    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"
Nov 03 21:11:34.621213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.621175    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" event={"ID":"413aa1c3-d204-4712-8f57-45641afa1b9b","Type":"ContainerDied","Data":"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"}
Nov 03 21:11:34.621305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.621218    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r" event={"ID":"413aa1c3-d204-4712-8f57-45641afa1b9b","Type":"ContainerDied","Data":"74de3bc97cef226d1ee07a9be6745742c6e1d8c4493f17814d16f6c67c45bece"}
Nov 03 21:11:34.622694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.622674    5915 generic.go:356] "Generic (PLEG): container finished" podID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerID="a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00" exitCode=0
Nov 03 21:11:34.622776 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.622722    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"
Nov 03 21:11:34.622776 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.622754    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" event={"ID":"fbf276c4-45a5-4f4d-9742-73576ce2e479","Type":"ContainerDied","Data":"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"}
Nov 03 21:11:34.622855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.622793    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9" event={"ID":"fbf276c4-45a5-4f4d-9742-73576ce2e479","Type":"ContainerDied","Data":"f8990c4f98c63ccd67301c69ac932957d468db801692eeabf8bc035c0aa397bf"}
Nov 03 21:11:34.629424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.629407    5915 scope.go:122] "RemoveContainer" containerID="5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"
Nov 03 21:11:34.629690 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:34.629671    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01\": container with ID starting with 5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01 not found: ID does not exist" containerID="5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"
Nov 03 21:11:34.629762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.629698    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01"} err="failed to get container status \"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01\": rpc error: code = NotFound desc = could not find container \"5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01\": container with ID starting with 5950d4af22d7d4540157554d89fb4d3ed590b1fac6ea1ca47ff4cd7e50939f01 not found: ID does not exist"
Nov 03 21:11:34.629762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.629714    5915 scope.go:122] "RemoveContainer" containerID="d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"
Nov 03 21:11:34.639447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.639424    5915 scope.go:122] "RemoveContainer" containerID="d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"
Nov 03 21:11:34.639749 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:34.639728    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273\": container with ID starting with d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273 not found: ID does not exist" containerID="d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"
Nov 03 21:11:34.639852 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.639759    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273"} err="failed to get container status \"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273\": rpc error: code = NotFound desc = could not find container \"d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273\": container with ID starting with d8d0e7947eee71c7d8521b909e39c5c1eec4cb0b920cc9646e95b1eb82918273 not found: ID does not exist"
Nov 03 21:11:34.639852 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.639780    5915 scope.go:122] "RemoveContainer" containerID="a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"
Nov 03 21:11:34.650015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.649996    5915 scope.go:122] "RemoveContainer" containerID="a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"
Nov 03 21:11:34.650247 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:34.650229    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00\": container with ID starting with a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00 not found: ID does not exist" containerID="a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"
Nov 03 21:11:34.650317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.650254    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00"} err="failed to get container status \"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00\": rpc error: code = NotFound desc = could not find container \"a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00\": container with ID starting with a80957452237c4c00ff9c207b210213482da36f9076044275393df54d8211d00 not found: ID does not exist"
Nov 03 21:11:34.686236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686196    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.686417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686255    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.686417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686286    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.686417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686345    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.686417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686378    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.686417 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686409    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.686727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686467    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.686727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686496    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.686727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686502    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.686727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686528    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.686949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686789    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.687002 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.686987    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.687079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687059    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687142    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687177    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.687221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687201    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.687221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687214    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687254    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.687369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687287    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687338    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.687369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687354    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687401    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.687591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687436    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.687591 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687475    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687592    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687619    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687658    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687689    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687724    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.687762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687753    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\") pod \"413aa1c3-d204-4712-8f57-45641afa1b9b\" (UID: \"413aa1c3-d204-4712-8f57-45641afa1b9b\") "
Nov 03 21:11:34.688124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687790    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\") pod \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\" (UID: \"9e8190bf-c316-4f3a-bafe-517a8b3254fb\") "
Nov 03 21:11:34.688124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.687854    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\") pod \"fbf276c4-45a5-4f4d-9742-73576ce2e479\" (UID: \"fbf276c4-45a5-4f4d-9742-73576ce2e479\") "
Nov 03 21:11:34.688399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.688380    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.688472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.688406    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.688472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.688419    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.688472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.688434    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.688472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.688448    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.689829 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.689763    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.689829 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.689764    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.690111 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.690085    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.690361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.690337    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.691911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.691885    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.692111 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.692053    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.692220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.692197    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "kube-api-access-ctg82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.692291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.692257    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.692726    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.692925    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693009    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "kube-api-access-ff22b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693217    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693285    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693454    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693493    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.693976 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.693839    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.694500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.694449    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.694500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.694408    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo" pod "413aa1c3-d204-4712-8f57-45641afa1b9b" (UID: "413aa1c3-d204-4712-8f57-45641afa1b9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.695119 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.695082    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.695447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.695422    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "kube-api-access-rczdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.695746 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.695724    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo" pod "fbf276c4-45a5-4f4d-9742-73576ce2e479" (UID: "fbf276c4-45a5-4f4d-9742-73576ce2e479"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.697385 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.697352    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo" pod "9e8190bf-c316-4f3a-bafe-517a8b3254fb" (UID: "9e8190bf-c316-4f3a-bafe-517a8b3254fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.789736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789705    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.789736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789729    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.789736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789739    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.789736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789748    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/413aa1c3-d204-4712-8f57-45641afa1b9b-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789760    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789776    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789786    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789795    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbf276c4-45a5-4f4d-9742-73576ce2e479-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789803    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789812    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789821    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctg82\" (UniqueName: \"kubernetes.io/projected/9e8190bf-c316-4f3a-bafe-517a8b3254fb-kube-api-access-ctg82\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789831    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789843    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789857    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff22b\" (UniqueName: \"kubernetes.io/projected/413aa1c3-d204-4712-8f57-45641afa1b9b-kube-api-access-ff22b\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789866    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbf276c4-45a5-4f4d-9742-73576ce2e479-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789877    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789885    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rczdv\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-kube-api-access-rczdv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789893    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789901    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789910    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbf276c4-45a5-4f4d-9742-73576ce2e479-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789919    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8190bf-c316-4f3a-bafe-517a8b3254fb-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.790006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:34.789932    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/413aa1c3-d204-4712-8f57-45641afa1b9b-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:35.190744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.190711    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 21:11:35.194088 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.194057    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-vfb4r"]
Nov 03 21:11:35.307567 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.307519    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:11:35.315393 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.315331    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-v26v9"]
Nov 03 21:11:35.419224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.419145    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 21:11:35.424315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.424288    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-nrm2g"]
Nov 03 21:11:35.682152 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.682080    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" path="/var/lib/kubelet/pods/413aa1c3-d204-4712-8f57-45641afa1b9b/volumes"
Nov 03 21:11:35.682606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.682590    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" path="/var/lib/kubelet/pods/4adbdd4a-ba51-4c24-a2f9-421056d099ca/volumes"
Nov 03 21:11:35.683043 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.683031    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" path="/var/lib/kubelet/pods/9e8190bf-c316-4f3a-bafe-517a8b3254fb/volumes"
Nov 03 21:11:35.683867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:35.683850    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" path="/var/lib/kubelet/pods/fbf276c4-45a5-4f4d-9742-73576ce2e479/volumes"
Nov 03 21:11:36.513844 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:36.513791    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:37.004700 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:37.004665    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:11:40.076585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:40.076525    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:11:40.188089 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:40.188057    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:11:40.452321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:40.452291    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:40.601530 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:40.601500    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:41.339990 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:41.339954    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:42.173935 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:42.173905    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:42.675244 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:42.675211    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:45.077524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.077491    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" podUID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" containerName="payload-processing" containerID="cri-o://1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512" gracePeriod=25
Nov 03 21:11:45.188337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.188299    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" podUID="3b6c3ae2-33be-457a-8177-092699dcd3fa" containerName="payload-pre-processing" containerID="cri-o://4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5" gracePeriod=25
Nov 03 21:11:45.359244 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.359215    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:11:45.445665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.445624    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\") pod \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") "
Nov 03 21:11:45.445838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.445726    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\" (UniqueName: \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\") pod \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\" (UID: \"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e\") "
Nov 03 21:11:45.446023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.446000    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume" pod "6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" (UID: "6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:45.446157 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.446137    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:45.448031 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.448003    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2" pod "6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" (UID: "6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e"). InnerVolumeSpecName "kube-api-access-x2km2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:45.467158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.467137    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:11:45.546519 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.546484    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\") pod \"3b6c3ae2-33be-457a-8177-092699dcd3fa\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") "
Nov 03 21:11:45.546723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.546620    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\" (UniqueName: \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\") pod \"3b6c3ae2-33be-457a-8177-092699dcd3fa\" (UID: \"3b6c3ae2-33be-457a-8177-092699dcd3fa\") "
Nov 03 21:11:45.546893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.546865    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume" pod "3b6c3ae2-33be-457a-8177-092699dcd3fa" (UID: "3b6c3ae2-33be-457a-8177-092699dcd3fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:45.547149 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.547129    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2km2\" (UniqueName: \"kubernetes.io/projected/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e-kube-api-access-x2km2\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:45.547216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.547154    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b6c3ae2-33be-457a-8177-092699dcd3fa-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:45.548913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.548892    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5" pod "3b6c3ae2-33be-457a-8177-092699dcd3fa" (UID: "3b6c3ae2-33be-457a-8177-092699dcd3fa"). InnerVolumeSpecName "kube-api-access-qrdp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:45.648054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.647979    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrdp5\" (UniqueName: \"kubernetes.io/projected/3b6c3ae2-33be-457a-8177-092699dcd3fa-kube-api-access-qrdp5\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:45.725682 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.725652    5915 generic.go:356] "Generic (PLEG): container finished" podID="3b6c3ae2-33be-457a-8177-092699dcd3fa" containerID="4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5" exitCode=0
Nov 03 21:11:45.725832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.725723    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"
Nov 03 21:11:45.725832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.725721    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" event={"ID":"3b6c3ae2-33be-457a-8177-092699dcd3fa","Type":"ContainerDied","Data":"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"}
Nov 03 21:11:45.725832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.725770    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl" event={"ID":"3b6c3ae2-33be-457a-8177-092699dcd3fa","Type":"ContainerDied","Data":"45e7086cc6c2d3207dc9b7ecabdb271f5155758a0b2afb276b09623d401e3e7e"}
Nov 03 21:11:45.725832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.725792    5915 scope.go:122] "RemoveContainer" containerID="4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"
Nov 03 21:11:45.727454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.727432    5915 generic.go:356] "Generic (PLEG): container finished" podID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" containerID="1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512" exitCode=0
Nov 03 21:11:45.727601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.727478    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" event={"ID":"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e","Type":"ContainerDied","Data":"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"}
Nov 03 21:11:45.727601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.727494    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"
Nov 03 21:11:45.727601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.727519    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh" event={"ID":"6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e","Type":"ContainerDied","Data":"b58add992769e60c26e84d4d7da9cd005baa02774eb30d05be426b700832be0e"}
Nov 03 21:11:45.737811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.737793    5915 scope.go:122] "RemoveContainer" containerID="4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"
Nov 03 21:11:45.738086 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:45.738061    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5\": container with ID starting with 4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5 not found: ID does not exist" containerID="4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"
Nov 03 21:11:45.738203 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.738097    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5"} err="failed to get container status \"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5\": rpc error: code = NotFound desc = could not find container \"4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5\": container with ID starting with 4105b47882c6fade05293802d192c081e12a5030f69394b554a58d1bfa372aa5 not found: ID does not exist"
Nov 03 21:11:45.738203 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.738120    5915 scope.go:122] "RemoveContainer" containerID="1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"
Nov 03 21:11:45.747523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.747506    5915 scope.go:122] "RemoveContainer" containerID="1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"
Nov 03 21:11:45.747811 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:45.747792    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512\": container with ID starting with 1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512 not found: ID does not exist" containerID="1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"
Nov 03 21:11:45.747921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.747816    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512"} err="failed to get container status \"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512\": rpc error: code = NotFound desc = could not find container \"1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512\": container with ID starting with 1cded4a541d972ef2894358bddda7ad198b4b8027bbc6bfc05e44201a6b14512 not found: ID does not exist"
Nov 03 21:11:45.892535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.892504    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:11:45.896908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:45.896883    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-218976-588f58c694-47hvh"]
Nov 03 21:11:46.006001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:46.005971    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:11:46.009228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:46.009206    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-218976-7745f8b655-jk2gl"]
Nov 03 21:11:46.176716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:46.176685    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:47.681654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:47.681614    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b6c3ae2-33be-457a-8177-092699dcd3fa" path="/var/lib/kubelet/pods/3b6c3ae2-33be-457a-8177-092699dcd3fa/volumes"
Nov 03 21:11:47.682225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:47.682204    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" path="/var/lib/kubelet/pods/6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e/volumes"
Nov 03 21:11:50.717317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717284    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:11:50.717911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717895    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.717952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717913    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.717952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717928    5915 cpu_manager.go:414] "containerMap: removing container" podUID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" containerName="payload-processing"
Nov 03 21:11:50.717952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717933    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" containerName="payload-processing"
Nov 03 21:11:50.717952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717948    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.717952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717953    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717961    5915 cpu_manager.go:414] "containerMap: removing container" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717966    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717979    5915 cpu_manager.go:414] "containerMap: removing container" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.717985    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718006    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3b6c3ae2-33be-457a-8177-092699dcd3fa" containerName="payload-pre-processing"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718011    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b6c3ae2-33be-457a-8177-092699dcd3fa" containerName="payload-pre-processing"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718019    5915 cpu_manager.go:414] "containerMap: removing container" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718024    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy"
Nov 03 21:11:50.718146 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718139    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718161    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d2ddfbe-f153-4ab2-9bc1-36aaa7f48b1e" containerName="payload-processing"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718176    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbdd4a-ba51-4c24-a2f9-421056d099ca" containerName="istio-proxy"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718182    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbf276c4-45a5-4f4d-9742-73576ce2e479" containerName="istio-proxy"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718192    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e8190bf-c316-4f3a-bafe-517a8b3254fb" containerName="istio-proxy"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718203    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b6c3ae2-33be-457a-8177-092699dcd3fa" containerName="payload-pre-processing"
Nov 03 21:11:50.718424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.718209    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="413aa1c3-d204-4712-8f57-45641afa1b9b" containerName="istio-proxy"
Nov 03 21:11:50.723200 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.723172    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.777523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.777500    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-01c7a3-serving-cert\""
Nov 03 21:11:50.781921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.781896    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:11:50.814490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.814459    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.814645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.814584    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.814645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.814617    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqnc\" (UniqueName: \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.814772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.814716    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.916376 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.916345    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.916573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.916413    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.916573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.916432    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqnc\" (UniqueName: \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.916573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.916490    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.916573 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:50.916525    5915 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-serving-cert: secret "maas-api-e2e-shared-b-w2-01c7a3-serving-cert" not found
Nov 03 21:11:50.916798 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:11:50.916611    5915 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls podName:b569d67c-d232-4f02-8f27-231a287d680f nodeName:}" failed. No retries permitted until 2026-11-03 21:11:51.416591409 +0000 UTC m=+1925.867960580 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls") pod "maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" (UID: "b569d67c-d232-4f02-8f27-231a287d680f") : secret "maas-api-e2e-shared-b-w2-01c7a3-serving-cert" not found
Nov 03 21:11:50.917260 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.917239    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.919173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.919152    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:50.996047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:50.995953    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:51.006021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.005894    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqnc\" (UniqueName: \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:51.170369 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.170334    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:51.341927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.341829    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:51.423861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.423831    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:51.426354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.426329    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:51.637755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:51.637682    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:52.019914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:52.019888    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:11:52.021208 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:11:52.021184    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb569d67c_d232_4f02_8f27_231a287d680f.slice/crio-0d640d3ec3f5dc627d4bc07bb3ed3c696103b029fd8758250d41ceca798c2301 WatchSource:0}: Error finding container 0d640d3ec3f5dc627d4bc07bb3ed3c696103b029fd8758250d41ceca798c2301: Status 404 returned error can't find the container with id 0d640d3ec3f5dc627d4bc07bb3ed3c696103b029fd8758250d41ceca798c2301
Nov 03 21:11:52.790151 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:52.790113    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" event={"ID":"b569d67c-d232-4f02-8f27-231a287d680f","Type":"ContainerStarted","Data":"0d640d3ec3f5dc627d4bc07bb3ed3c696103b029fd8758250d41ceca798c2301"}
Nov 03 21:11:53.811086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:53.810166    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" event={"ID":"b569d67c-d232-4f02-8f27-231a287d680f","Type":"ContainerStarted","Data":"f572e5c5b85594cfc0d0c6a714448bd566fb71c4e5f33a340a2cc64f4f211178"}
Nov 03 21:11:53.811086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:53.810306    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:11:53.930943 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:53.930887    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" podStartSLOduration=3.017144838 podStartE2EDuration="3.930873791s" totalImagesPullingTime="913.728953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:11:53.927482876 +0000 UTC m=+1928.378852055" watchObservedRunningTime="2026-11-03 21:11:53.930873791 +0000 UTC m=+1928.382242968"
Nov 03 21:11:55.971805 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:55.971766    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:11:59.822679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:11:59.822652    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:12:00.825720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:00.825687    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:05.713738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:05.713702    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:07.005245 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:07.005204    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:12:10.516116 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:10.516081    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:15.256987 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:15.256953    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:18.205281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:18.205245    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:20.008788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:20.008752    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:25.073584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:25.073526    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:25.235860 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:25.235831    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:29.600255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:29.591969    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:12:29.600255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:29.599358    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" podUID="a3b04b00-64e7-473f-91b7-323e491ede5a" containerName="maas-api" containerID="cri-o://abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982" gracePeriod=30
Nov 03 21:12:29.697952 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:29.697918    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:29.918472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:29.918449    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:12:30.053580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.049160    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\" (UniqueName: \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\") pod \"a3b04b00-64e7-473f-91b7-323e491ede5a\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") "
Nov 03 21:12:30.053580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.049263    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") pod \"a3b04b00-64e7-473f-91b7-323e491ede5a\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") "
Nov 03 21:12:30.053580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.049422    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\") pod \"a3b04b00-64e7-473f-91b7-323e491ede5a\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") "
Nov 03 21:12:30.053580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.049454    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\") pod \"a3b04b00-64e7-473f-91b7-323e491ede5a\" (UID: \"a3b04b00-64e7-473f-91b7-323e491ede5a\") "
Nov 03 21:12:30.053580 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.050467    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca" pod "a3b04b00-64e7-473f-91b7-323e491ede5a" (UID: "a3b04b00-64e7-473f-91b7-323e491ede5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:30.064090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.064058    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs" pod "a3b04b00-64e7-473f-91b7-323e491ede5a" (UID: "a3b04b00-64e7-473f-91b7-323e491ede5a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:30.078052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.077780    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls" pod "a3b04b00-64e7-473f-91b7-323e491ede5a" (UID: "a3b04b00-64e7-473f-91b7-323e491ede5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:30.085769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.084982    5915 generic.go:356] "Generic (PLEG): container finished" podID="a3b04b00-64e7-473f-91b7-323e491ede5a" containerID="abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982" exitCode=0
Nov 03 21:12:30.085769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.085275    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" event={"ID":"a3b04b00-64e7-473f-91b7-323e491ede5a","Type":"ContainerDied","Data":"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"}
Nov 03 21:12:30.085769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.085328    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh" event={"ID":"a3b04b00-64e7-473f-91b7-323e491ede5a","Type":"ContainerDied","Data":"12824a5927869232c396e756cd9d107f81a201e23ceadbf1d15502f54c904e40"}
Nov 03 21:12:30.085769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.085357    5915 scope.go:122] "RemoveContainer" containerID="abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"
Nov 03 21:12:30.085769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.085507    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"
Nov 03 21:12:30.094765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.094736    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59" pod "a3b04b00-64e7-473f-91b7-323e491ede5a" (UID: "a3b04b00-64e7-473f-91b7-323e491ede5a"). InnerVolumeSpecName "kube-api-access-wvf59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:30.106263 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.106241    5915 scope.go:122] "RemoveContainer" containerID="abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"
Nov 03 21:12:30.106622 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:30.106602    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982\": container with ID starting with abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982 not found: ID does not exist" containerID="abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"
Nov 03 21:12:30.106703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.106631    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982"} err="failed to get container status \"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982\": rpc error: code = NotFound desc = could not find container \"abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982\": container with ID starting with abaddc0b366d6ca3197596a3e50e87a95bbda9b0e8262d96015728ba9a559982 not found: ID does not exist"
Nov 03 21:12:30.151906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.151828    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:30.151906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.151866    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3b04b00-64e7-473f-91b7-323e491ede5a-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:30.151906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.151878    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvf59\" (UniqueName: \"kubernetes.io/projected/a3b04b00-64e7-473f-91b7-323e491ede5a-kube-api-access-wvf59\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:30.151906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.151887    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3b04b00-64e7-473f-91b7-323e491ede5a-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:30.506276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.506240    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:12:30.536247 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:30.536216    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-218976-7c79578767-2qtfh"]
Nov 03 21:12:31.491237 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:31.491179    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:31.682037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:31.682007    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3b04b00-64e7-473f-91b7-323e491ede5a" path="/var/lib/kubelet/pods/a3b04b00-64e7-473f-91b7-323e491ede5a/volumes"
Nov 03 21:12:31.737921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:31.737876    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:31.893502 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:31.893424    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:32.453143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:32.453112    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:35.118502 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:35.118461    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:37.004661 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:37.004623    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:12:37.077730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:37.077687    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:39.820897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:39.820854    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:40.064428 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.064391    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:40.153595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.153489    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:12:40.153770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.153745    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" containerID="cri-o://f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1" gracePeriod=30
Nov 03 21:12:40.215758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.215728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:40.387038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.387001    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:40.541911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.541870    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:40.723475 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:40.723446    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:41.276652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:41.276613    5915 patch_prober.go:29] interesting pod/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:12:41.277025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:41.276693    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:12:42.477060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:42.477001    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:42.631371 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:42.631336    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:45.433501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.433440    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:12:45.577195 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577172    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577208    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577289    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577308    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577327    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577368    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577436    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:45.577616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577528    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577596    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577598    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:45.577890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577627    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\") pod \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\" (UID: \"cfeabfcb-aa53-41fd-861a-a4ab990de3a4\") "
Nov 03 21:12:45.577890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577727    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:45.577890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577765    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.577996    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.578098    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.578119    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.578133    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.578155    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.578225 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.578169    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.579883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.579832    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:45.579883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.579839    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "kube-api-access-hnnds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:45.579883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.579873    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:12:45.580048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.579878    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token" pod "cfeabfcb-aa53-41fd-861a-a4ab990de3a4" (UID: "cfeabfcb-aa53-41fd-861a-a4ab990de3a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:45.655728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.655691    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 21:12:45.679833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.679804    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.679833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.679831    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.680047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.679840    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnnds\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-kube-api-access-hnnds\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.680047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.679851    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfeabfcb-aa53-41fd-861a-a4ab990de3a4-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:45.756024 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:45.755999    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:46.222427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.222392    5915 generic.go:356] "Generic (PLEG): container finished" podID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerID="f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1" exitCode=0
Nov 03 21:12:46.222615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.222465    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"
Nov 03 21:12:46.222615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.222481    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerDied","Data":"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"}
Nov 03 21:12:46.222615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.222542    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t" event={"ID":"cfeabfcb-aa53-41fd-861a-a4ab990de3a4","Type":"ContainerDied","Data":"b9df0854a9623617498f744c9a0dced9ed9056dc5b5e6e1d08070e97790a178b"}
Nov 03 21:12:46.222615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.222598    5915 scope.go:122] "RemoveContainer" containerID="f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"
Nov 03 21:12:46.233215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.233197    5915 scope.go:122] "RemoveContainer" containerID="cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"
Nov 03 21:12:46.242864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.242835    5915 scope.go:122] "RemoveContainer" containerID="f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"
Nov 03 21:12:46.243134 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:46.243117    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1\": container with ID starting with f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1 not found: ID does not exist" containerID="f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"
Nov 03 21:12:46.243180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.243144    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1"} err="failed to get container status \"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1\": rpc error: code = NotFound desc = could not find container \"f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1\": container with ID starting with f5fed62fe458bfe2a495e9f6f3edef9377deb5f2862953ce91ed85d0fa6223e1 not found: ID does not exist"
Nov 03 21:12:46.243180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.243161    5915 scope.go:122] "RemoveContainer" containerID="cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"
Nov 03 21:12:46.243420 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:46.243403    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d\": container with ID starting with cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d not found: ID does not exist" containerID="cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"
Nov 03 21:12:46.243461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.243425    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d"} err="failed to get container status \"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d\": rpc error: code = NotFound desc = could not find container \"cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d\": container with ID starting with cc6d893b8be28c2375f777583827b56e0ab903ae2758737c3b66a2bbab38d92d not found: ID does not exist"
Nov 03 21:12:46.389223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.389185    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:12:46.438501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:46.438469    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-218976-openshift-default-79647dd6b5-qwf9t"]
Nov 03 21:12:47.325552 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:47.325521    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:47.473515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:47.473476    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:12:47.603425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:47.603335    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:12:47.681675 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:47.681641    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" path="/var/lib/kubelet/pods/cfeabfcb-aa53-41fd-861a-a4ab990de3a4/volumes"
Nov 03 21:12:47.864681 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:47.864594    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:48.046940 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:48.046905    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:50.056681 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:50.056644    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:12:50.169615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:50.169588    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:12:50.443663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:50.443589    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:50.589722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:50.589692    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:50.836177 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:50.836143    5915 scope.go:122] "RemoveContainer" containerID="5e52fb4b50ec9139e314666a3c472a635f37f704fd5f31fe418cfd2307f2759a"
Nov 03 21:12:52.261065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.261037    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:52.474719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.474679    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerName="payload-processing" containerID="cri-o://f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858" gracePeriod=25
Nov 03 21:12:52.499029 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:12:52.499009    5915 logging.go:55] [core] [Channel #2504 SubChannel #2505]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.132:50052", ServerName: "10.132.0.132:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.132:50052: connect: connection refused"
Nov 03 21:12:52.604590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.604540    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" podUID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" containerName="payload-pre-processing" containerID="cri-o://26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c" gracePeriod=25
Nov 03 21:12:52.737256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.737229    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:12:52.800854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.800243    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:52.817906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.817866    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\") pod \"49ae2ca8-3685-4a42-b996-5f408a0f1942\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") "
Nov 03 21:12:52.823796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.819699    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume" pod "49ae2ca8-3685-4a42-b996-5f408a0f1942" (UID: "49ae2ca8-3685-4a42-b996-5f408a0f1942"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:52.824140 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.824111    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\" (UniqueName: \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\") pod \"49ae2ca8-3685-4a42-b996-5f408a0f1942\" (UID: \"49ae2ca8-3685-4a42-b996-5f408a0f1942\") "
Nov 03 21:12:52.824930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.824903    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49ae2ca8-3685-4a42-b996-5f408a0f1942-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:52.827848 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.827145    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw" pod "49ae2ca8-3685-4a42-b996-5f408a0f1942" (UID: "49ae2ca8-3685-4a42-b996-5f408a0f1942"). InnerVolumeSpecName "kube-api-access-64nsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:52.876624 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.876601    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:12:52.925963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.925934    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\") pod \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") "
Nov 03 21:12:52.926183 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.925985    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\" (UniqueName: \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\") pod \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\" (UID: \"5517b19e-6acb-4b24-8525-18fa2d9ee8a5\") "
Nov 03 21:12:52.926408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.926377    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume" pod "5517b19e-6acb-4b24-8525-18fa2d9ee8a5" (UID: "5517b19e-6acb-4b24-8525-18fa2d9ee8a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:52.926535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.926444    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64nsw\" (UniqueName: \"kubernetes.io/projected/49ae2ca8-3685-4a42-b996-5f408a0f1942-kube-api-access-64nsw\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:52.928477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.928456    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt" pod "5517b19e-6acb-4b24-8525-18fa2d9ee8a5" (UID: "5517b19e-6acb-4b24-8525-18fa2d9ee8a5"). InnerVolumeSpecName "kube-api-access-jpktt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:52.945519 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:52.945498    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:53.028085 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.028053    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:53.028085 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.028082    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpktt\" (UniqueName: \"kubernetes.io/projected/5517b19e-6acb-4b24-8525-18fa2d9ee8a5-kube-api-access-jpktt\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:53.277740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.277710    5915 generic.go:356] "Generic (PLEG): container finished" podID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" containerID="26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c" exitCode=0
Nov 03 21:12:53.278235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.277764    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"
Nov 03 21:12:53.278235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.277792    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" event={"ID":"5517b19e-6acb-4b24-8525-18fa2d9ee8a5","Type":"ContainerDied","Data":"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"}
Nov 03 21:12:53.278235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.277868    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g" event={"ID":"5517b19e-6acb-4b24-8525-18fa2d9ee8a5","Type":"ContainerDied","Data":"79adc7d2efc492e4b499a554b8607b7b90342b7fc1dec73ba4f8e28fdcc87d27"}
Nov 03 21:12:53.278235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.277880    5915 scope.go:122] "RemoveContainer" containerID="26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"
Nov 03 21:12:53.279342 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.279321    5915 generic.go:356] "Generic (PLEG): container finished" podID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerID="f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858" exitCode=0
Nov 03 21:12:53.279457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.279399    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" event={"ID":"49ae2ca8-3685-4a42-b996-5f408a0f1942","Type":"ContainerDied","Data":"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"}
Nov 03 21:12:53.279457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.279454    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" event={"ID":"49ae2ca8-3685-4a42-b996-5f408a0f1942","Type":"ContainerDied","Data":"1ee25ef709fa88fd3dfdedd6ad82fe80ac387bff5483902bdfc0be8b93a4a461"}
Nov 03 21:12:53.279623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.279604    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"
Nov 03 21:12:53.290397 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.290380    5915 scope.go:122] "RemoveContainer" containerID="26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"
Nov 03 21:12:53.290689 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:53.290668    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c\": container with ID starting with 26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c not found: ID does not exist" containerID="26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"
Nov 03 21:12:53.290791 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.290697    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c"} err="failed to get container status \"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c\": rpc error: code = NotFound desc = could not find container \"26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c\": container with ID starting with 26e90e53abd7bb674a3fe4ceaada8ee97d1a68a306e41019e987eef67ee9a37c not found: ID does not exist"
Nov 03 21:12:53.290791 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.290714    5915 scope.go:122] "RemoveContainer" containerID="f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"
Nov 03 21:12:53.304488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.304468    5915 scope.go:122] "RemoveContainer" containerID="f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"
Nov 03 21:12:53.305071 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:53.305051    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858\": container with ID starting with f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858 not found: ID does not exist" containerID="f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"
Nov 03 21:12:53.305157 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.305079    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858"} err="failed to get container status \"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858\": rpc error: code = NotFound desc = could not find container \"f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858\": container with ID starting with f2d1730c3e818df0f5d70412df9b99634183e649cae0ef63843fe6ea0bad0858 not found: ID does not exist"
Nov 03 21:12:53.444526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.444490    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:12:53.448719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.448692    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w6-70fe43-7c46b56d75-r6p9g"]
Nov 03 21:12:53.498885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.498855    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.132:50052\" within 1s: context deadline exceeded"
Nov 03 21:12:53.555965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.555887    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:12:53.559516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.559490    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-70fe43-57f974c866-4n742"]
Nov 03 21:12:53.682385 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.682345    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" path="/var/lib/kubelet/pods/49ae2ca8-3685-4a42-b996-5f408a0f1942/volumes"
Nov 03 21:12:53.682882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:53.682861    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" path="/var/lib/kubelet/pods/5517b19e-6acb-4b24-8525-18fa2d9ee8a5/volumes"
Nov 03 21:12:55.057105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.057072    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" podUID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" containerName="payload-processing" containerID="cri-o://959187b8b20ea8d0cadb8115421075c7e996af0f12dd3123ad5a9506fbafaeda" gracePeriod=25
Nov 03 21:12:55.170356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.170246    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" podUID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" containerName="payload-pre-processing" containerID="cri-o://572628db07ccc903ab33fdf2c3846ba8c85f192e3f13902464cab51857ea8c48" gracePeriod=25
Nov 03 21:12:55.300277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.300249    5915 generic.go:356] "Generic (PLEG): container finished" podID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" containerID="959187b8b20ea8d0cadb8115421075c7e996af0f12dd3123ad5a9506fbafaeda" exitCode=0
Nov 03 21:12:55.300421 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.300331    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" event={"ID":"ea8a6173-0413-453f-a8ff-ae42cd0871bf","Type":"ContainerDied","Data":"959187b8b20ea8d0cadb8115421075c7e996af0f12dd3123ad5a9506fbafaeda"}
Nov 03 21:12:55.302267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.302244    5915 generic.go:356] "Generic (PLEG): container finished" podID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" containerID="572628db07ccc903ab33fdf2c3846ba8c85f192e3f13902464cab51857ea8c48" exitCode=0
Nov 03 21:12:55.302381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.302286    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" event={"ID":"03bbb487-1cfa-47c0-ba3e-91579d7ef660","Type":"ContainerDied","Data":"572628db07ccc903ab33fdf2c3846ba8c85f192e3f13902464cab51857ea8c48"}
Nov 03 21:12:55.333832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.333671    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:55.359007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.358979    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:12:55.454211 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.454185    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\") pod \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") "
Nov 03 21:12:55.454355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.454223    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\" (UniqueName: \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\") pod \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\" (UID: \"ea8a6173-0413-453f-a8ff-ae42cd0871bf\") "
Nov 03 21:12:55.454586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.454546    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume" pod "ea8a6173-0413-453f-a8ff-ae42cd0871bf" (UID: "ea8a6173-0413-453f-a8ff-ae42cd0871bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:55.456391 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.456366    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8" pod "ea8a6173-0413-453f-a8ff-ae42cd0871bf" (UID: "ea8a6173-0413-453f-a8ff-ae42cd0871bf"). InnerVolumeSpecName "kube-api-access-276r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:55.461938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.461919    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:12:55.496579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.494372    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:55.554974 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.554944    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\") pod \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") "
Nov 03 21:12:55.555172 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.554983    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\" (UniqueName: \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\") pod \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\" (UID: \"03bbb487-1cfa-47c0-ba3e-91579d7ef660\") "
Nov 03 21:12:55.555375 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.555352    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume" pod "03bbb487-1cfa-47c0-ba3e-91579d7ef660" (UID: "03bbb487-1cfa-47c0-ba3e-91579d7ef660"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:55.555532 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.555515    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03bbb487-1cfa-47c0-ba3e-91579d7ef660-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:55.555621 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.555539    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea8a6173-0413-453f-a8ff-ae42cd0871bf-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:55.555621 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.555570    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-276r8\" (UniqueName: \"kubernetes.io/projected/ea8a6173-0413-453f-a8ff-ae42cd0871bf-kube-api-access-276r8\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:55.557226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.557207    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf" pod "03bbb487-1cfa-47c0-ba3e-91579d7ef660" (UID: "03bbb487-1cfa-47c0-ba3e-91579d7ef660"). InnerVolumeSpecName "kube-api-access-6dwdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:55.654246 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.645006    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:55.657841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:55.657779    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dwdf\" (UniqueName: \"kubernetes.io/projected/03bbb487-1cfa-47c0-ba3e-91579d7ef660-kube-api-access-6dwdf\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:56.312141 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.312117    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"
Nov 03 21:12:56.312597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.312149    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59" event={"ID":"ea8a6173-0413-453f-a8ff-ae42cd0871bf","Type":"ContainerDied","Data":"3f511208eac3c0ec4953a584fb2c08e2c46fda9465e42258cfb7cfefee845113"}
Nov 03 21:12:56.312597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.312209    5915 scope.go:122] "RemoveContainer" containerID="959187b8b20ea8d0cadb8115421075c7e996af0f12dd3123ad5a9506fbafaeda"
Nov 03 21:12:56.313949 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.313912    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8" event={"ID":"03bbb487-1cfa-47c0-ba3e-91579d7ef660","Type":"ContainerDied","Data":"ba429c6e7694fdc4437f8ed41cac69c2e930967f68c971d5af0c4407932fc0c9"}
Nov 03 21:12:56.314081 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.313927    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"
Nov 03 21:12:56.323080 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.323065    5915 scope.go:122] "RemoveContainer" containerID="572628db07ccc903ab33fdf2c3846ba8c85f192e3f13902464cab51857ea8c48"
Nov 03 21:12:56.481392 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.481352    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:12:56.530799 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.530767    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-2f36ee-7789bc547b-h5s59"]
Nov 03 21:12:56.583632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.580212    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:12:56.590026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.589985    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" podUID="20f0ed39-f25a-4fe7-8346-c8616e20633d" containerName="maas-api" containerID="cri-o://72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4" gracePeriod=30
Nov 03 21:12:56.637065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.637026    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:12:56.640665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.640638    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-2f36ee-7cdd4f9cbc-94mf8"]
Nov 03 21:12:56.713592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.713531    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:56.937943 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:56.937915    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:12:57.080891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.080860    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\" (UniqueName: \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\") pod \"20f0ed39-f25a-4fe7-8346-c8616e20633d\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") "
Nov 03 21:12:57.080891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.080902    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\") pod \"20f0ed39-f25a-4fe7-8346-c8616e20633d\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") "
Nov 03 21:12:57.081159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.080996    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") pod \"20f0ed39-f25a-4fe7-8346-c8616e20633d\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") "
Nov 03 21:12:57.081159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.081093    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\") pod \"20f0ed39-f25a-4fe7-8346-c8616e20633d\" (UID: \"20f0ed39-f25a-4fe7-8346-c8616e20633d\") "
Nov 03 21:12:57.081423 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.081392    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca" pod "20f0ed39-f25a-4fe7-8346-c8616e20633d" (UID: "20f0ed39-f25a-4fe7-8346-c8616e20633d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:57.081745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.081717    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f0ed39-f25a-4fe7-8346-c8616e20633d-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:57.083510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.083480    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb" pod "20f0ed39-f25a-4fe7-8346-c8616e20633d" (UID: "20f0ed39-f25a-4fe7-8346-c8616e20633d"). InnerVolumeSpecName "kube-api-access-jc5nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:57.083992 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.083961    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls" pod "20f0ed39-f25a-4fe7-8346-c8616e20633d" (UID: "20f0ed39-f25a-4fe7-8346-c8616e20633d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:57.084102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.084031    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs" pod "20f0ed39-f25a-4fe7-8346-c8616e20633d" (UID: "20f0ed39-f25a-4fe7-8346-c8616e20633d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:57.198121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.198027    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc5nb\" (UniqueName: \"kubernetes.io/projected/20f0ed39-f25a-4fe7-8346-c8616e20633d-kube-api-access-jc5nb\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:57.198392 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.198369    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:57.198517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.198488    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f0ed39-f25a-4fe7-8346-c8616e20633d-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:57.322975 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.322939    5915 generic.go:356] "Generic (PLEG): container finished" podID="20f0ed39-f25a-4fe7-8346-c8616e20633d" containerID="72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4" exitCode=0
Nov 03 21:12:57.323410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.322986    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"
Nov 03 21:12:57.323410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.323026    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" event={"ID":"20f0ed39-f25a-4fe7-8346-c8616e20633d","Type":"ContainerDied","Data":"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"}
Nov 03 21:12:57.323410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.323111    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8" event={"ID":"20f0ed39-f25a-4fe7-8346-c8616e20633d","Type":"ContainerDied","Data":"d4f45251f765b38564d9ad192baea2bbe59c5e1379d1969af506240d084d6ed8"}
Nov 03 21:12:57.323410 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.323142    5915 scope.go:122] "RemoveContainer" containerID="72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"
Nov 03 21:12:57.334021 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.334002    5915 scope.go:122] "RemoveContainer" containerID="72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"
Nov 03 21:12:57.334288 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:12:57.334270    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4\": container with ID starting with 72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4 not found: ID does not exist" containerID="72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"
Nov 03 21:12:57.334333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.334297    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4"} err="failed to get container status \"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4\": rpc error: code = NotFound desc = could not find container \"72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4\": container with ID starting with 72f6b5d12ca62e5e5f4ceb76159b7d31372b4d60bbcaf93aed470fdf00daacc4 not found: ID does not exist"
Nov 03 21:12:57.491259 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.491221    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:12:57.498702 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.498672    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-2f36ee-6bb76df87d-n82z8"]
Nov 03 21:12:57.681835 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.681803    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" path="/var/lib/kubelet/pods/03bbb487-1cfa-47c0-ba3e-91579d7ef660/volumes"
Nov 03 21:12:57.682340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.682324    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f0ed39-f25a-4fe7-8346-c8616e20633d" path="/var/lib/kubelet/pods/20f0ed39-f25a-4fe7-8346-c8616e20633d/volumes"
Nov 03 21:12:57.682745 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:57.682730    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" path="/var/lib/kubelet/pods/ea8a6173-0413-453f-a8ff-ae42cd0871bf/volumes"
Nov 03 21:12:58.127199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.127165    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:12:58.127427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.127387    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" containerName="maas-api" containerID="cri-o://d35afeb2d5ddc4a64c6a7066b7d1babaa0da5f909e17d7777201c1802251d3c9" gracePeriod=30
Nov 03 21:12:58.205511 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.205473    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.139:8443/health\": dial tcp 10.132.0.139:8443: connect: connection refused"
Nov 03 21:12:58.228082 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.228052    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:58.341473 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.341435    5915 generic.go:356] "Generic (PLEG): container finished" podID="3a5d712e-7942-46cc-b776-a106bf237a92" containerID="d35afeb2d5ddc4a64c6a7066b7d1babaa0da5f909e17d7777201c1802251d3c9" exitCode=0
Nov 03 21:12:58.342124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.341611    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" event={"ID":"3a5d712e-7942-46cc-b776-a106bf237a92","Type":"ContainerDied","Data":"d35afeb2d5ddc4a64c6a7066b7d1babaa0da5f909e17d7777201c1802251d3c9"}
Nov 03 21:12:58.379797 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.379721    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:58.409056 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.409034    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:12:58.524592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.522146    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\" (UniqueName: \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\") pod \"3a5d712e-7942-46cc-b776-a106bf237a92\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") "
Nov 03 21:12:58.524592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.522209    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") pod \"3a5d712e-7942-46cc-b776-a106bf237a92\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") "
Nov 03 21:12:58.524592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.522264    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\") pod \"3a5d712e-7942-46cc-b776-a106bf237a92\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") "
Nov 03 21:12:58.524592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.522369    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\") pod \"3a5d712e-7942-46cc-b776-a106bf237a92\" (UID: \"3a5d712e-7942-46cc-b776-a106bf237a92\") "
Nov 03 21:12:58.524592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.524132    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca" pod "3a5d712e-7942-46cc-b776-a106bf237a92" (UID: "3a5d712e-7942-46cc-b776-a106bf237a92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:58.560592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.554165    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls" pod "3a5d712e-7942-46cc-b776-a106bf237a92" (UID: "3a5d712e-7942-46cc-b776-a106bf237a92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:58.562057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.562017    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425" pod "3a5d712e-7942-46cc-b776-a106bf237a92" (UID: "3a5d712e-7942-46cc-b776-a106bf237a92"). InnerVolumeSpecName "kube-api-access-2m425". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:58.565087 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.565042    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs" pod "3a5d712e-7942-46cc-b776-a106bf237a92" (UID: "3a5d712e-7942-46cc-b776-a106bf237a92"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:12:58.624541 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.624503    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m425\" (UniqueName: \"kubernetes.io/projected/3a5d712e-7942-46cc-b776-a106bf237a92-kube-api-access-2m425\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:58.624541 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.624541    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:58.624802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.624570    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5d712e-7942-46cc-b776-a106bf237a92-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:58.624802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.624599    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a5d712e-7942-46cc-b776-a106bf237a92-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:58.630220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:58.630148    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:59.333000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.332965    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:12:59.355665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.355629    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr" event={"ID":"3a5d712e-7942-46cc-b776-a106bf237a92","Type":"ContainerDied","Data":"19f463e4dc1b66c7504cacc012f891889ecf5a849f92dbd8eb58184902f9dbd1"}
Nov 03 21:12:59.356098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.355687    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"
Nov 03 21:12:59.356098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.355708    5915 scope.go:122] "RemoveContainer" containerID="d35afeb2d5ddc4a64c6a7066b7d1babaa0da5f909e17d7777201c1802251d3c9"
Nov 03 21:12:59.524448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.524408    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:12:59.574357 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.574327    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-70fe43-6d7b5d55df-qpfdr"]
Nov 03 21:12:59.687972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.687933    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" path="/var/lib/kubelet/pods/3a5d712e-7942-46cc-b776-a106bf237a92/volumes"
Nov 03 21:12:59.808748 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:12:59.808671    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:00.125741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:00.125668    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:00.373240 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:00.373207    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:00.677456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:00.677426    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main" containerID="cri-o://e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657" gracePeriod=45
Nov 03 21:13:00.926311 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:00.926274    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:00.961654 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:00.961625    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:13:01.056754 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.056729    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\" (UniqueName: \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.056898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.056866    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.056898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.056888    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.057024 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.056949    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\" (UniqueName: \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.057024 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.056994    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.057134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.057047    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\") pod \"18276f96-22f2-463b-96c2-7b09d2dd9914\" (UID: \"18276f96-22f2-463b-96c2-7b09d2dd9914\") "
Nov 03 21:13:01.057358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.057301    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:01.057544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.057365    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:01.057544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.057424    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:01.059605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.059579    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:01.059837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.059782    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:01.059837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.059791    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx" pod "18276f96-22f2-463b-96c2-7b09d2dd9914" (UID: "18276f96-22f2-463b-96c2-7b09d2dd9914"). InnerVolumeSpecName "kube-api-access-l2nzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:01.158348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158316    5915 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-home\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.158348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158347    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2nzx\" (UniqueName: \"kubernetes.io/projected/18276f96-22f2-463b-96c2-7b09d2dd9914-kube-api-access-l2nzx\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.158606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158360    5915 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-dshm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.158606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158369    5915 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-model-cache\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.158606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158378    5915 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18276f96-22f2-463b-96c2-7b09d2dd9914-tls-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.158606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.158386    5915 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18276f96-22f2-463b-96c2-7b09d2dd9914-tmp-dir\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.375185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.375148    5915 generic.go:356] "Generic (PLEG): container finished" podID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerID="e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657" exitCode=0
Nov 03 21:13:01.375648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.375202    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" event={"ID":"18276f96-22f2-463b-96c2-7b09d2dd9914","Type":"ContainerDied","Data":"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"}
Nov 03 21:13:01.375648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.375216    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"
Nov 03 21:13:01.375648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.375273    5915 scope.go:122] "RemoveContainer" containerID="e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"
Nov 03 21:13:01.375648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.375259    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s" event={"ID":"18276f96-22f2-463b-96c2-7b09d2dd9914","Type":"ContainerDied","Data":"d4e862c1aba368c28d9a22ac30c971cada6c66557fa3a8f8254971007a401664"}
Nov 03 21:13:01.389001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.388979    5915 scope.go:122] "RemoveContainer" containerID="e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"
Nov 03 21:13:01.389324 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:01.389304    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657\": container with ID starting with e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657 not found: ID does not exist" containerID="e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"
Nov 03 21:13:01.389393 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.389333    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657"} err="failed to get container status \"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657\": rpc error: code = NotFound desc = could not find container \"e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657\": container with ID starting with e1493c989a45f2615e64b352f0c33522831f01f05d769915e404c97412890657 not found: ID does not exist"
Nov 03 21:13:01.544315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.544285    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 21:13:01.547295 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.547269    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/premium-w1-d01cef-kserve-549c56cf8d-gtt7s"]
Nov 03 21:13:01.681959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:01.681890    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" path="/var/lib/kubelet/pods/18276f96-22f2-463b-96c2-7b09d2dd9914/volumes"
Nov 03 21:13:05.457700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:05.457664    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:05.709527 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:05.709439    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:05.878008 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:05.877974    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:06.079116 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:06.079035    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:06.111312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:06.111277    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:13:06.111617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:06.111575    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy" containerID="cri-o://e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a" gracePeriod=30
Nov 03 21:13:07.009690 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:07.009648    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:13:08.061496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.061467    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:08.139802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.139765    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:13:08.224306 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.224272    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:13:08.228360 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.228334    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:08.401297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.401222    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:13:08.401504 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.401477    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" containerID="cri-o://a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da" gracePeriod=30
Nov 03 21:13:08.442986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.442524    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:08.635233 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.633801    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:08.790184 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:08.790153    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:10.633365 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:10.633330    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:11.155192 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.155159    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:13:11.155333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.155224    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:13:11.373628 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.373600    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:13:11.469514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.469485    5915 generic.go:356] "Generic (PLEG): container finished" podID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerID="e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a" exitCode=0
Nov 03 21:13:11.469723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.469540    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"
Nov 03 21:13:11.469723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.469594    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerDied","Data":"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"}
Nov 03 21:13:11.469723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.469651    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g" event={"ID":"ad9a34e6-a541-4cba-b714-a76243c87f38","Type":"ContainerDied","Data":"062fec9530ddf16b3a5c66129c69ecb2a5a64e607b191a329d4e9c98a7c869fb"}
Nov 03 21:13:11.469723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.469674    5915 scope.go:122] "RemoveContainer" containerID="e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"
Nov 03 21:13:11.480293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.480276    5915 scope.go:122] "RemoveContainer" containerID="02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"
Nov 03 21:13:11.489697 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.489676    5915 scope.go:122] "RemoveContainer" containerID="e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"
Nov 03 21:13:11.489960 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:11.489944    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a\": container with ID starting with e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a not found: ID does not exist" containerID="e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"
Nov 03 21:13:11.490022 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.489973    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a"} err="failed to get container status \"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a\": rpc error: code = NotFound desc = could not find container \"e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a\": container with ID starting with e2458a9f68e203724c7cd7c66b4fea0e87e32413354227482ccd740c8ea1bc5a not found: ID does not exist"
Nov 03 21:13:11.490022 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.489992    5915 scope.go:122] "RemoveContainer" containerID="02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"
Nov 03 21:13:11.490240 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:11.490210    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43\": container with ID starting with 02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43 not found: ID does not exist" containerID="02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"
Nov 03 21:13:11.490282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.490245    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43"} err="failed to get container status \"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43\": rpc error: code = NotFound desc = could not find container \"02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43\": container with ID starting with 02cf6eae8a58b60f079fb0c83107392c802940b46e0641147411b6761a46ba43 not found: ID does not exist"
Nov 03 21:13:11.494549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494529    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494670 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494648    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494696    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494731    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494843 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494784    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494843 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494816    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494900    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.494954 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494813    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:11.495064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494951    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.495064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.494968    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:11.495064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495001    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\") pod \"ad9a34e6-a541-4cba-b714-a76243c87f38\" (UID: \"ad9a34e6-a541-4cba-b714-a76243c87f38\") "
Nov 03 21:13:11.495227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495056    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:11.495227 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495201    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:11.495340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495229    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:11.495491 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495471    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.495593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495499    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.495593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495513    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad9a34e6-a541-4cba-b714-a76243c87f38-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.495593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495526    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.495593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.495541    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.497183 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.497161    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:13:11.497392 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.497376    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:11.497474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.497388    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "kube-api-access-wtqxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:11.497474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.497434    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy" pod "ad9a34e6-a541-4cba-b714-a76243c87f38" (UID: "ad9a34e6-a541-4cba-b714-a76243c87f38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:11.596570 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.596532    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.596729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.596579    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtqxq\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-kube-api-access-wtqxq\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.596729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.596593    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.596729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.596602    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad9a34e6-a541-4cba-b714-a76243c87f38-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:11.884881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.884808    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:13:11.887394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:11.887374    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-2f36ee-openshift-default-747694c48c-xqw9g"]
Nov 03 21:13:13.140095 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.140038    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" podUID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" containerName="payload-processing" containerID="cri-o://9736ece8fb0090d83fc153c3d6e530d4addce34ae6ba17f6e95266111318e34a" gracePeriod=25
Nov 03 21:13:13.224713 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.224673    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" podUID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" containerName="payload-pre-processing" containerID="cri-o://449e13c54fa1eb11debe9fd7a42c8a6b797ce87930954adf9583ca58dbc0b092" gracePeriod=25
Nov 03 21:13:13.489288 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.489253    5915 generic.go:356] "Generic (PLEG): container finished" podID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" containerID="449e13c54fa1eb11debe9fd7a42c8a6b797ce87930954adf9583ca58dbc0b092" exitCode=0
Nov 03 21:13:13.490697 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.489353    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" event={"ID":"b6678936-ca03-4f7a-888d-23bb2cb87cb4","Type":"ContainerDied","Data":"449e13c54fa1eb11debe9fd7a42c8a6b797ce87930954adf9583ca58dbc0b092"}
Nov 03 21:13:13.491406 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.491387    5915 generic.go:356] "Generic (PLEG): container finished" podID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" containerID="9736ece8fb0090d83fc153c3d6e530d4addce34ae6ba17f6e95266111318e34a" exitCode=0
Nov 03 21:13:13.491535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.491424    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" event={"ID":"296c205e-4ebf-4b9c-be50-1b9b98ea5598","Type":"ContainerDied","Data":"9736ece8fb0090d83fc153c3d6e530d4addce34ae6ba17f6e95266111318e34a"}
Nov 03 21:13:13.526289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.526184    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:13:13.530335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.530307    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:13:13.627310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627279    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\") pod \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") "
Nov 03 21:13:13.627513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627354    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\") pod \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") "
Nov 03 21:13:13.627513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627397    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\" (UniqueName: \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\") pod \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\" (UID: \"296c205e-4ebf-4b9c-be50-1b9b98ea5598\") "
Nov 03 21:13:13.627513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627477    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\" (UniqueName: \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\") pod \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\" (UID: \"b6678936-ca03-4f7a-888d-23bb2cb87cb4\") "
Nov 03 21:13:13.627788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627765    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume" pod "296c205e-4ebf-4b9c-be50-1b9b98ea5598" (UID: "296c205e-4ebf-4b9c-be50-1b9b98ea5598"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:13.627788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627772    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume" pod "b6678936-ca03-4f7a-888d-23bb2cb87cb4" (UID: "b6678936-ca03-4f7a-888d-23bb2cb87cb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:13.627973 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627951    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/296c205e-4ebf-4b9c-be50-1b9b98ea5598-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.628041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.627981    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6678936-ca03-4f7a-888d-23bb2cb87cb4-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.629931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.629906    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w" pod "b6678936-ca03-4f7a-888d-23bb2cb87cb4" (UID: "b6678936-ca03-4f7a-888d-23bb2cb87cb4"). InnerVolumeSpecName "kube-api-access-hw69w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:13.630197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.630180    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g" pod "296c205e-4ebf-4b9c-be50-1b9b98ea5598" (UID: "296c205e-4ebf-4b9c-be50-1b9b98ea5598"). InnerVolumeSpecName "kube-api-access-2xw8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:13.677455 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.677377    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:13:13.681445 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.681420    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" path="/var/lib/kubelet/pods/ad9a34e6-a541-4cba-b714-a76243c87f38/volumes"
Nov 03 21:13:13.728834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.728806    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw69w\" (UniqueName: \"kubernetes.io/projected/b6678936-ca03-4f7a-888d-23bb2cb87cb4-kube-api-access-hw69w\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.728834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.728833    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xw8g\" (UniqueName: \"kubernetes.io/projected/296c205e-4ebf-4b9c-be50-1b9b98ea5598-kube-api-access-2xw8g\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.829478 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829444    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829511    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829569    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829607    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829640    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829674    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.829752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829713    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.830096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829773    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:13.830096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829781    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.830096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829864    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\") pod \"93500355-da00-4f11-b925-11b8553bd9e6\" (UID: \"93500355-da00-4f11-b925-11b8553bd9e6\") "
Nov 03 21:13:13.830096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829917    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:13.830096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.829975    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:13.830476 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830154    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:13.830476 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830165    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:13.830822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830801    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.830911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830826    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.830911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830843    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93500355-da00-4f11-b925-11b8553bd9e6-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.830911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830858    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.830911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.830873    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.832178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.832153    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:13.832276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.832180    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "kube-api-access-qbdfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:13.832276 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.832185    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:13:13.832380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.832366    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token" pod "93500355-da00-4f11-b925-11b8553bd9e6" (UID: "93500355-da00-4f11-b925-11b8553bd9e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:13.932583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.932478    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93500355-da00-4f11-b925-11b8553bd9e6-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.932583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.932504    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93500355-da00-4f11-b925-11b8553bd9e6-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.932583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.932514    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:13.932583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:13.932523    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbdfm\" (UniqueName: \"kubernetes.io/projected/93500355-da00-4f11-b925-11b8553bd9e6-kube-api-access-qbdfm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:14.499383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.499355    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"
Nov 03 21:13:14.499904 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.499405    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7" event={"ID":"296c205e-4ebf-4b9c-be50-1b9b98ea5598","Type":"ContainerDied","Data":"e2d49f058ae91aa67166dbb65fb56d2dbef68bb99f198be6fb0a428a90f506a9"}
Nov 03 21:13:14.499904 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.499473    5915 scope.go:122] "RemoveContainer" containerID="9736ece8fb0090d83fc153c3d6e530d4addce34ae6ba17f6e95266111318e34a"
Nov 03 21:13:14.502448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.501722    5915 generic.go:356] "Generic (PLEG): container finished" podID="93500355-da00-4f11-b925-11b8553bd9e6" containerID="a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da" exitCode=0
Nov 03 21:13:14.502448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.501789    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"
Nov 03 21:13:14.502448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.501853    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerDied","Data":"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"}
Nov 03 21:13:14.502448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.501898    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2" event={"ID":"93500355-da00-4f11-b925-11b8553bd9e6","Type":"ContainerDied","Data":"7109e39af32746c16d69cdbfe2e148ad022cb9b9aca676b8310baf36d820830b"}
Nov 03 21:13:14.503836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.503812    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"
Nov 03 21:13:14.503980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.503843    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v" event={"ID":"b6678936-ca03-4f7a-888d-23bb2cb87cb4","Type":"ContainerDied","Data":"abd4f2cc48999ad3e9258876b3a1ceac988f836db8aa8828c70bd5ca0c2faab8"}
Nov 03 21:13:14.513306 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.513292    5915 scope.go:122] "RemoveContainer" containerID="a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"
Nov 03 21:13:14.523414 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.523388    5915 scope.go:122] "RemoveContainer" containerID="db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"
Nov 03 21:13:14.533208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.533175    5915 scope.go:122] "RemoveContainer" containerID="a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"
Nov 03 21:13:14.533631 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:14.533611    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da\": container with ID starting with a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da not found: ID does not exist" containerID="a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"
Nov 03 21:13:14.533714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.533642    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da"} err="failed to get container status \"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da\": rpc error: code = NotFound desc = could not find container \"a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da\": container with ID starting with a03bdec60f9a8c57e1665e3ba272c417fc62a857cb3464fcc3364ec237d073da not found: ID does not exist"
Nov 03 21:13:14.533714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.533659    5915 scope.go:122] "RemoveContainer" containerID="db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"
Nov 03 21:13:14.533955 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:14.533937    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e\": container with ID starting with db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e not found: ID does not exist" containerID="db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"
Nov 03 21:13:14.534015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.533966    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e"} err="failed to get container status \"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e\": rpc error: code = NotFound desc = could not find container \"db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e\": container with ID starting with db2dc97af0a37edfb1386db620bb4ed829c74eeb3605ba366c64ff0579a5c75e not found: ID does not exist"
Nov 03 21:13:14.534015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.533992    5915 scope.go:122] "RemoveContainer" containerID="449e13c54fa1eb11debe9fd7a42c8a6b797ce87930954adf9583ca58dbc0b092"
Nov 03 21:13:14.671191 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.671153    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:13:14.721420 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.721379    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-70fe43-openshift-default-64545cdc8f-flnw2"]
Nov 03 21:13:14.784705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.784617    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:13:14.795904 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.795874    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-b8ac68-78f7fb8467-xhd6v"]
Nov 03 21:13:14.912718 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.912685    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:13:14.917896 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.917866    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-b8ac68-6c46f56c8d-vskx7"]
Nov 03 21:13:14.930168 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.930142    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:13:14.930363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:14.930344    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" podUID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" containerName="maas-api" containerID="cri-o://93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30" gracePeriod=30
Nov 03 21:13:15.032610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.032576    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:15.263707 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.263678    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:13:15.364932 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.364843    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\") pod \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") "
Nov 03 21:13:15.365271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.365238    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\" (UniqueName: \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\") pod \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") "
Nov 03 21:13:15.365399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.365299    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") pod \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") "
Nov 03 21:13:15.365399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.365337    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\") pod \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\" (UID: \"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73\") "
Nov 03 21:13:15.366258 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.366230    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca" pod "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" (UID: "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:15.369711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.369688    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w" pod "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" (UID: "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73"). InnerVolumeSpecName "kube-api-access-lc65w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:15.369869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.369760    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs" pod "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" (UID: "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:15.370760 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.370735    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls" pod "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" (UID: "ee97cf83-6b4f-4eb0-ad31-1586a0e02b73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:15.466678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.466645    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc65w\" (UniqueName: \"kubernetes.io/projected/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-kube-api-access-lc65w\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:15.466678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.466674    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:15.466678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.466683    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:15.466918 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.466693    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:15.533582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.515531    5915 generic.go:356] "Generic (PLEG): container finished" podID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" containerID="93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30" exitCode=0
Nov 03 21:13:15.533582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.515642    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" event={"ID":"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73","Type":"ContainerDied","Data":"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"}
Nov 03 21:13:15.533582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.515695    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw" event={"ID":"ee97cf83-6b4f-4eb0-ad31-1586a0e02b73","Type":"ContainerDied","Data":"1872b1268317d432cfb58a119c3f1c0ffd8839795928e3b7634f89c5cd823853"}
Nov 03 21:13:15.533582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.515724    5915 scope.go:122] "RemoveContainer" containerID="93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"
Nov 03 21:13:15.533582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.515839    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"
Nov 03 21:13:15.573898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.573872    5915 scope.go:122] "RemoveContainer" containerID="93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"
Nov 03 21:13:15.574233 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:15.574210    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30\": container with ID starting with 93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30 not found: ID does not exist" containerID="93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"
Nov 03 21:13:15.574293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.574247    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30"} err="failed to get container status \"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30\": rpc error: code = NotFound desc = could not find container \"93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30\": container with ID starting with 93150ad24808be167ab1b8279477ac8bcc03a1ddf70685113413386314bdeb30 not found: ID does not exist"
Nov 03 21:13:15.681231 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.681136    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" path="/var/lib/kubelet/pods/296c205e-4ebf-4b9c-be50-1b9b98ea5598/volumes"
Nov 03 21:13:15.681730 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.681708    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93500355-da00-4f11-b925-11b8553bd9e6" path="/var/lib/kubelet/pods/93500355-da00-4f11-b925-11b8553bd9e6/volumes"
Nov 03 21:13:15.682278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.682256    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" path="/var/lib/kubelet/pods/b6678936-ca03-4f7a-888d-23bb2cb87cb4/volumes"
Nov 03 21:13:15.686834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.686805    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:13:15.689338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:15.689316    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b8ac68-75644fdc58-dzbtw"]
Nov 03 21:13:16.586445 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:16.586384    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:16.821336 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:16.821289    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:16.985849 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:16.985803    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:17.580416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:17.580386    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:17.680862 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:17.680827    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" path="/var/lib/kubelet/pods/ee97cf83-6b4f-4eb0-ad31-1586a0e02b73/volumes"
Nov 03 21:13:18.011770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:18.011730    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:18.208627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:18.201455    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:18.583686 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:18.583657    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:20.664785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:20.664748    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:20.866587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:20.858957    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:22.286722 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:22.286688    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:22.453415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:22.453384    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:25.937181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:25.937084    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:26.131105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:26.131069    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:26.312252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:26.312216    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:26.504051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:26.504018    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:27.351891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:27.351863    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:32.005941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:32.005909    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:32.167456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:32.167369    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:32.317782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:32.317701    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:36.357144 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:36.357108    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:36.658158 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:36.658074    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:13:36.658335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:36.658295    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" containerID="cri-o://f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd" gracePeriod=30
Nov 03 21:13:36.678572 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:36.678534    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:37.002883 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:37.002845    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:13:37.413033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.412961    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:13:37.413459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.413145    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" podUID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" containerName="maas-api" containerID="cri-o://1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f" gracePeriod=30
Nov 03 21:13:37.566583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.566526    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:37.725440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.725408    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:13:37.727296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.727263    5915 generic.go:356] "Generic (PLEG): container finished" podID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" containerID="1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f" exitCode=0
Nov 03 21:13:37.727430 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.727342    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" event={"ID":"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e","Type":"ContainerDied","Data":"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"}
Nov 03 21:13:37.727430 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.727414    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv" event={"ID":"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e","Type":"ContainerDied","Data":"33a57c043ecd2ec02264a9487bf5b6cb8a2e729cc1ac5485713d34498d0a3827"}
Nov 03 21:13:37.727517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.727446    5915 scope.go:122] "RemoveContainer" containerID="1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"
Nov 03 21:13:37.737729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.737708    5915 scope.go:122] "RemoveContainer" containerID="1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"
Nov 03 21:13:37.738007 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:37.737986    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f\": container with ID starting with 1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f not found: ID does not exist" containerID="1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"
Nov 03 21:13:37.738091 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.738015    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f"} err="failed to get container status \"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f\": rpc error: code = NotFound desc = could not find container \"1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f\": container with ID starting with 1e77582a6ded1f22036185c4838633a90ed90d10c7c351c77076fdd26f2fd34f not found: ID does not exist"
Nov 03 21:13:37.761470 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.761448    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\" (UniqueName: \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\") pod \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") "
Nov 03 21:13:37.761617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.761543    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") pod \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") "
Nov 03 21:13:37.761617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.761602    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\") pod \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") "
Nov 03 21:13:37.761728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.761650    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\") pod \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\" (UID: \"d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e\") "
Nov 03 21:13:37.762006 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.761982    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca" pod "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" (UID: "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:37.763984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.763955    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs" pod "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" (UID: "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:37.763984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.763972    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls" pod "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" (UID: "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:37.764122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.764046    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l" pod "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" (UID: "d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e"). InnerVolumeSpecName "kube-api-access-br89l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:37.863692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.863652    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br89l\" (UniqueName: \"kubernetes.io/projected/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-kube-api-access-br89l\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:37.863692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.863692    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:37.863883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.863709    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:37.863883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:37.863723    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:38.048744 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:38.048689    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:38.391017 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:38.390935    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:38.734870 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:38.734845    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"
Nov 03 21:13:38.898819 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:38.898787    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:13:38.948162 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:38.948131    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-32c41e-7c54ffb964-cz7jv"]
Nov 03 21:13:39.019449 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:39.019371    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:39.193459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:39.193417    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:39.426699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:39.426603    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:39.587445 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:39.587410    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:39.680911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:39.680839    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" path="/var/lib/kubelet/pods/d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e/volumes"
Nov 03 21:13:40.249195 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:40.249167    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:41.594663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:41.594623    5915 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:13:41.595165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:41.594703    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:13:41.932876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:41.932851    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:13:42.004887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.004861    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.004887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.004891    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.004947    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.004963    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005023    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005097 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005073    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005107    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005143    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005145    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:42.005251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005185    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\") pod \"fb672755-dd4e-430d-af31-4da8f757e02c\" (UID: \"fb672755-dd4e-430d-af31-4da8f757e02c\") "
Nov 03 21:13:42.005449 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005275    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:42.005512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005492    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:42.005653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005632    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:42.005911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005886    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:42.005961 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005852    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.005961 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005944    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.006028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005962    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb672755-dd4e-430d-af31-4da8f757e02c-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.006028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.005977    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.007423 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.007314    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:42.007526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.007488    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "kube-api-access-6xp24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:42.007875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.007842    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:13:42.008078 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.008057    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token" pod "fb672755-dd4e-430d-af31-4da8f757e02c" (UID: "fb672755-dd4e-430d-af31-4da8f757e02c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:42.107280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.107237    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb672755-dd4e-430d-af31-4da8f757e02c-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.107280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.107274    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.107280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.107284    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.107280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.107293    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xp24\" (UniqueName: \"kubernetes.io/projected/fb672755-dd4e-430d-af31-4da8f757e02c-kube-api-access-6xp24\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.107585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.107303    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb672755-dd4e-430d-af31-4da8f757e02c-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:42.764072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.764035    5915 generic.go:356] "Generic (PLEG): container finished" podID="fb672755-dd4e-430d-af31-4da8f757e02c" containerID="f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd" exitCode=0
Nov 03 21:13:42.764618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.764107    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"
Nov 03 21:13:42.764618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.764116    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerDied","Data":"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"}
Nov 03 21:13:42.764618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.764177    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx" event={"ID":"fb672755-dd4e-430d-af31-4da8f757e02c","Type":"ContainerDied","Data":"19acdfffdbd70d6f204a92f71706aac6df5f6e509e4367b5a48bccbe2a3b916b"}
Nov 03 21:13:42.764618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.764210    5915 scope.go:122] "RemoveContainer" containerID="f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"
Nov 03 21:13:42.775941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.775798    5915 scope.go:122] "RemoveContainer" containerID="bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"
Nov 03 21:13:42.784806 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.784788    5915 scope.go:122] "RemoveContainer" containerID="f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"
Nov 03 21:13:42.785059 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:42.785039    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd\": container with ID starting with f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd not found: ID does not exist" containerID="f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"
Nov 03 21:13:42.785147 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.785071    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd"} err="failed to get container status \"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd\": rpc error: code = NotFound desc = could not find container \"f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd\": container with ID starting with f20947d6d7065d402017d4ef4fd56346ea913d2157ca3b85f9a225eae9ea61cd not found: ID does not exist"
Nov 03 21:13:42.785147 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.785098    5915 scope.go:122] "RemoveContainer" containerID="bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"
Nov 03 21:13:42.785360 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:42.785343    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b\": container with ID starting with bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b not found: ID does not exist" containerID="bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"
Nov 03 21:13:42.785408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.785366    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b"} err="failed to get container status \"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b\": rpc error: code = NotFound desc = could not find container \"bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b\": container with ID starting with bed998ad58af0c08a9a566fd61523f4c6cf6f8f6ea2016b918bfd8249da15a8b not found: ID does not exist"
Nov 03 21:13:42.859783 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.859740    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:42.929531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.929501    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:13:42.979409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:42.979378    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b8ac68-openshift-default-b4b84574c-6d8tx"]
Nov 03 21:13:43.367011 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:43.366963    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:43.681129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:43.681061    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" path="/var/lib/kubelet/pods/fb672755-dd4e-430d-af31-4da8f757e02c/volumes"
Nov 03 21:13:43.872167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:43.872138    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:13:43.872618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:43.872324    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" podUID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" containerName="maas-api" containerID="cri-o://7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4" gracePeriod=30
Nov 03 21:13:43.994166 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:43.994127    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:44.182913 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.182891    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:13:44.230007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.229980    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\") pod \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") "
Nov 03 21:13:44.230154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.230015    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\") pod \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") "
Nov 03 21:13:44.230154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.230037    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\" (UniqueName: \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\") pod \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") "
Nov 03 21:13:44.230154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.230057    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") pod \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\" (UID: \"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0\") "
Nov 03 21:13:44.230416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.230394    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca" pod "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" (UID: "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:44.232356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.232330    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs" pod "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" (UID: "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:44.232356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.232340    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9" pod "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" (UID: "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0"). InnerVolumeSpecName "kube-api-access-8dms9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:44.232472 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.232366    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls" pod "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" (UID: "018ac50c-c050-4ee3-b2d8-38d7b0dd89b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:44.331719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.331633    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:44.331719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.331665    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:44.331719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.331682    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dms9\" (UniqueName: \"kubernetes.io/projected/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-kube-api-access-8dms9\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:44.331719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.331694    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:44.779362 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.779326    5915 generic.go:356] "Generic (PLEG): container finished" podID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" containerID="7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4" exitCode=0
Nov 03 21:13:44.779529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.779371    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"
Nov 03 21:13:44.779529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.779379    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" event={"ID":"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0","Type":"ContainerDied","Data":"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"}
Nov 03 21:13:44.779529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.779431    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467" event={"ID":"018ac50c-c050-4ee3-b2d8-38d7b0dd89b0","Type":"ContainerDied","Data":"8e4b83ae281010c1fb249e513d3b43685e1f010a5e143678bdbc91ced67ad187"}
Nov 03 21:13:44.779529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.779461    5915 scope.go:122] "RemoveContainer" containerID="7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"
Nov 03 21:13:44.789876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.789859    5915 scope.go:122] "RemoveContainer" containerID="7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"
Nov 03 21:13:44.790146 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:13:44.790119    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4\": container with ID starting with 7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4 not found: ID does not exist" containerID="7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"
Nov 03 21:13:44.790192 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.790154    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4"} err="failed to get container status \"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4\": rpc error: code = NotFound desc = could not find container \"7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4\": container with ID starting with 7a821531ba88a6ed31e2047b65a3c2782822415d4b8fde5599f1f7edd4bd75c4 not found: ID does not exist"
Nov 03 21:13:44.971755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:44.971718    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:13:45.021125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:45.021099    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-1e5494-69899868d5-zz467"]
Nov 03 21:13:45.505586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:45.505527    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:45.655040 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:45.655008    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:45.683863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:45.683831    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" path="/var/lib/kubelet/pods/018ac50c-c050-4ee3-b2d8-38d7b0dd89b0/volumes"
Nov 03 21:13:45.812845 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:45.812752    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:46.009226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:46.009190    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:46.182325 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:46.182232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:46.542791 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:46.542754    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:46.871289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:46.871217    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:47.053699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:47.053660    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:47.318980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:47.318945    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:47.764531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:47.764499    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:50.727390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:50.727345    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:50.874353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:50.874324    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:51.019125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:51.019100    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:52.903936 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:52.903902    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:53.534032 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:53.533982    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:53.693295 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:53.693263    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:53.895474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:53.895381    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:54.387839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:54.387809    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:55.665337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.665305    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:13:55.665781 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.665524    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" podUID="b569d67c-d232-4f02-8f27-231a287d680f" containerName="maas-api" containerID="cri-o://f572e5c5b85594cfc0d0c6a714448bd566fb71c4e5f33a340a2cc64f4f211178" gracePeriod=30
Nov 03 21:13:55.778265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.778225    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:55.909751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.909714    5915 generic.go:356] "Generic (PLEG): container finished" podID="b569d67c-d232-4f02-8f27-231a287d680f" containerID="f572e5c5b85594cfc0d0c6a714448bd566fb71c4e5f33a340a2cc64f4f211178" exitCode=0
Nov 03 21:13:55.910086 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.910063    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" event={"ID":"b569d67c-d232-4f02-8f27-231a287d680f","Type":"ContainerDied","Data":"f572e5c5b85594cfc0d0c6a714448bd566fb71c4e5f33a340a2cc64f4f211178"}
Nov 03 21:13:55.966640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:55.966616    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:13:56.119922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.119170    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\") pod \"b569d67c-d232-4f02-8f27-231a287d680f\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") "
Nov 03 21:13:56.119922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.119242    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") pod \"b569d67c-d232-4f02-8f27-231a287d680f\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") "
Nov 03 21:13:56.119922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.119298    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\" (UniqueName: \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\") pod \"b569d67c-d232-4f02-8f27-231a287d680f\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") "
Nov 03 21:13:56.119922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.119326    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\") pod \"b569d67c-d232-4f02-8f27-231a287d680f\" (UID: \"b569d67c-d232-4f02-8f27-231a287d680f\") "
Nov 03 21:13:56.120892 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.120498    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca" pod "b569d67c-d232-4f02-8f27-231a287d680f" (UID: "b569d67c-d232-4f02-8f27-231a287d680f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:56.130517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.129729    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs" pod "b569d67c-d232-4f02-8f27-231a287d680f" (UID: "b569d67c-d232-4f02-8f27-231a287d680f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:56.130517 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.130188    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc" pod "b569d67c-d232-4f02-8f27-231a287d680f" (UID: "b569d67c-d232-4f02-8f27-231a287d680f"). InnerVolumeSpecName "kube-api-access-dtqnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:56.132625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.132551    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls" pod "b569d67c-d232-4f02-8f27-231a287d680f" (UID: "b569d67c-d232-4f02-8f27-231a287d680f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:56.221099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.221002    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:56.221099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.221039    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b569d67c-d232-4f02-8f27-231a287d680f-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:56.221099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.221055    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtqnc\" (UniqueName: \"kubernetes.io/projected/b569d67c-d232-4f02-8f27-231a287d680f-kube-api-access-dtqnc\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:56.221099 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.221072    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b569d67c-d232-4f02-8f27-231a287d680f-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:56.918047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.918020    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"
Nov 03 21:13:56.918496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.918054    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm" event={"ID":"b569d67c-d232-4f02-8f27-231a287d680f","Type":"ContainerDied","Data":"0d640d3ec3f5dc627d4bc07bb3ed3c696103b029fd8758250d41ceca798c2301"}
Nov 03 21:13:56.918496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:56.918120    5915 scope.go:122] "RemoveContainer" containerID="f572e5c5b85594cfc0d0c6a714448bd566fb71c4e5f33a340a2cc64f4f211178"
Nov 03 21:13:57.083543 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.083502    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:13:57.135986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.135948    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-01c7a3-87b98f8bb-rbrvm"]
Nov 03 21:13:57.416464 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.416433    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:57.682279 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.682182    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b569d67c-d232-4f02-8f27-231a287d680f" path="/var/lib/kubelet/pods/b569d67c-d232-4f02-8f27-231a287d680f/volumes"
Nov 03 21:13:57.704824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.704794    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:57.902499 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:57.902464    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:58.418074 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:58.418043    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:13:59.459538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:59.459501    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:13:59.508604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:59.508571    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 21:13:59.735544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:13:59.735447    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:00.122092 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:00.121990    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 21:14:00.377882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:00.377792    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:01.566345 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:01.566307    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:03.425856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:03.425824    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:05.123195 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.123136    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" podUID="48f1600e-7212-4d38-bc34-85ee90efebad" containerName="payload-pre-processing" containerID="cri-o://1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0" gracePeriod=25
Nov 03 21:14:05.294658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.294538    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:05.403379 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.403349    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 21:14:05.469801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.469766    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:05.554680 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.554627    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\" (UniqueName: \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\") pod \"48f1600e-7212-4d38-bc34-85ee90efebad\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") "
Nov 03 21:14:05.554863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.554695    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\") pod \"48f1600e-7212-4d38-bc34-85ee90efebad\" (UID: \"48f1600e-7212-4d38-bc34-85ee90efebad\") "
Nov 03 21:14:05.555922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.555826    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume" pod "48f1600e-7212-4d38-bc34-85ee90efebad" (UID: "48f1600e-7212-4d38-bc34-85ee90efebad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:05.561925 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.561896    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd" pod "48f1600e-7212-4d38-bc34-85ee90efebad" (UID: "48f1600e-7212-4d38-bc34-85ee90efebad"). InnerVolumeSpecName "kube-api-access-dl6hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:05.648594 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.648490    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:05.655995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.655974    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dl6hd\" (UniqueName: \"kubernetes.io/projected/48f1600e-7212-4d38-bc34-85ee90efebad-kube-api-access-dl6hd\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:05.656095 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.655997    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f1600e-7212-4d38-bc34-85ee90efebad-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:05.979108 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.979078    5915 generic.go:356] "Generic (PLEG): container finished" podID="48f1600e-7212-4d38-bc34-85ee90efebad" containerID="1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0" exitCode=0
Nov 03 21:14:05.979265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.979134    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"
Nov 03 21:14:05.979265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.979166    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" event={"ID":"48f1600e-7212-4d38-bc34-85ee90efebad","Type":"ContainerDied","Data":"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"}
Nov 03 21:14:05.979265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.979225    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp" event={"ID":"48f1600e-7212-4d38-bc34-85ee90efebad","Type":"ContainerDied","Data":"240d765d0ada05df121e9f1815818e90ce9bb266950b9a5b2dc9d6c965c3c45d"}
Nov 03 21:14:05.979265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.979250    5915 scope.go:122] "RemoveContainer" containerID="1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"
Nov 03 21:14:05.990040 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.990023    5915 scope.go:122] "RemoveContainer" containerID="1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"
Nov 03 21:14:05.990320 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:05.990302    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0\": container with ID starting with 1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0 not found: ID does not exist" containerID="1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"
Nov 03 21:14:05.990376 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:05.990329    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0"} err="failed to get container status \"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0\": rpc error: code = NotFound desc = could not find container \"1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0\": container with ID starting with 1f1b7d75b8832ca793bd4ba10c39a6264de383f6ba8f3e24a0f00a62a1ae5fc0 not found: ID does not exist"
Nov 03 21:14:06.147878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:06.147850    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 21:14:06.154340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:06.154308    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:06.196501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:06.196474    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-e22abf-964d9cdc5-7cqvp"]
Nov 03 21:14:07.004701 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:07.004663    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:14:07.680640 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:07.680602    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48f1600e-7212-4d38-bc34-85ee90efebad" path="/var/lib/kubelet/pods/48f1600e-7212-4d38-bc34-85ee90efebad/volumes"
Nov 03 21:14:08.062894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.062857    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"]
Nov 03 21:14:08.063383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063363    5915 cpu_manager.go:414] "containerMap: removing container" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063385    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063400    5915 cpu_manager.go:414] "containerMap: removing container" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063407    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063420    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063427    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063436    5915 cpu_manager.go:414] "containerMap: removing container" podUID="48f1600e-7212-4d38-bc34-85ee90efebad" containerName="payload-pre-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063443    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48f1600e-7212-4d38-bc34-85ee90efebad" containerName="payload-pre-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063455    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" containerName="payload-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063462    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" containerName="payload-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063480    5915 cpu_manager.go:414] "containerMap: removing container" podUID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" containerName="payload-pre-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063487    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" containerName="payload-pre-processing"
Nov 03 21:14:08.063493 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063496    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063503    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063517    5915 cpu_manager.go:414] "containerMap: removing container" podUID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063524    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063536    5915 cpu_manager.go:414] "containerMap: removing container" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063543    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063581    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063589    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063605    5915 cpu_manager.go:414] "containerMap: removing container" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063612    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063624    5915 cpu_manager.go:414] "containerMap: removing container" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063632    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063641    5915 cpu_manager.go:414] "containerMap: removing container" podUID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063648    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063663    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a3b04b00-64e7-473f-91b7-323e491ede5a" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063670    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3b04b00-64e7-473f-91b7-323e491ede5a" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063683    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063689    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063699    5915 cpu_manager.go:414] "containerMap: removing container" podUID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063705    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063720    5915 cpu_manager.go:414] "containerMap: removing container" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063727    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063747    5915 cpu_manager.go:414] "containerMap: removing container" podUID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063755    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063768    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b569d67c-d232-4f02-8f27-231a287d680f" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063775    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b569d67c-d232-4f02-8f27-231a287d680f" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063785    5915 cpu_manager.go:414] "containerMap: removing container" podUID="20f0ed39-f25a-4fe7-8346-c8616e20633d" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063792    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f0ed39-f25a-4fe7-8346-c8616e20633d" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063804    5915 cpu_manager.go:414] "containerMap: removing container" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063811    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063825    5915 cpu_manager.go:414] "containerMap: removing container" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063832    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063964    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="03bbb487-1cfa-47c0-ba3e-91579d7ef660" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063978    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee97cf83-6b4f-4eb0-ad31-1586a0e02b73" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063987    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="5517b19e-6acb-4b24-8525-18fa2d9ee8a5" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.063996    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="49ae2ca8-3685-4a42-b996-5f408a0f1942" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064008    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6678936-ca03-4f7a-888d-23bb2cb87cb4" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064019    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064028    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064036    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="d72c7317-46e2-4ba5-8bcb-e03b3e8ec57e" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064044    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3b04b00-64e7-473f-91b7-323e491ede5a" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064054    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a5d712e-7942-46cc-b776-a106bf237a92" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064064    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="018ac50c-c050-4ee3-b2d8-38d7b0dd89b0" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064073    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="18276f96-22f2-463b-96c2-7b09d2dd9914" containerName="main"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064082    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064091    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="296c205e-4ebf-4b9c-be50-1b9b98ea5598" containerName="payload-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064100    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b569d67c-d232-4f02-8f27-231a287d680f" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064109    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f0ed39-f25a-4fe7-8346-c8616e20633d" containerName="maas-api"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064117    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb672755-dd4e-430d-af31-4da8f757e02c" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064126    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064135    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="93500355-da00-4f11-b925-11b8553bd9e6" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064144    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfeabfcb-aa53-41fd-861a-a4ab990de3a4" containerName="istio-proxy"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064151    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="48f1600e-7212-4d38-bc34-85ee90efebad" containerName="payload-pre-processing"
Nov 03 21:14:08.064142 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.064159    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea8a6173-0413-453f-a8ff-ae42cd0871bf" containerName="payload-processing"
Nov 03 21:14:08.067885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.067861    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.167441 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.167402    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:14:08.170198 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.170169    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"]
Nov 03 21:14:08.173647 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.173625    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:14:08.182233 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.182208    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzqj\" (UniqueName: \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.182367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.182254    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.182367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.182350    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.242524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.242493    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:08.283736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.283697    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.283930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.283866    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzqj\" (UniqueName: \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.283930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.283906    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.284440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.284418    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.286374 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.286352    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.347184 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.347113    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzqj\" (UniqueName: \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\") pod \"maas-api-revoke-sub-d4e35f83-jtbgx\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.387019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.386989    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:08.446683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.446512    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:08.772857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.772828    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"]
Nov 03 21:14:08.774755 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:14:08.774722    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4912afc9_a302_4518_88e2_6a23c1539fe4.slice/crio-2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f WatchSource:0}: Error finding container 2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f: Status 404 returned error can't find the container with id 2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f
Nov 03 21:14:08.917741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:08.917709    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:09.003814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:09.003782    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx" event={"ID":"4912afc9-a302-4518-88e2-6a23c1539fe4","Type":"ContainerStarted","Data":"2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f"}
Nov 03 21:14:10.012677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:10.012595    5915 generic.go:356] "Generic (PLEG): container finished" podID="4912afc9-a302-4518-88e2-6a23c1539fe4" containerID="5b25ec8b64c92471ac83edc49eb22584e299949353c964fa6ce44b960603847c" exitCode=0
Nov 03 21:14:10.013052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:10.012680    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx" event={"ID":"4912afc9-a302-4518-88e2-6a23c1539fe4","Type":"ContainerDied","Data":"5b25ec8b64c92471ac83edc49eb22584e299949353c964fa6ce44b960603847c"}
Nov 03 21:14:11.109359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.109318    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 21:14:11.124132 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.124100    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 21:14:11.176390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.176363    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:11.202217 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.202187    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:11.321519 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.321442    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\") pod \"4912afc9-a302-4518-88e2-6a23c1539fe4\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") "
Nov 03 21:14:11.321721 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.321525    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\" (UniqueName: \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\") pod \"4912afc9-a302-4518-88e2-6a23c1539fe4\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") "
Nov 03 21:14:11.321721 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.321615    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\") pod \"4912afc9-a302-4518-88e2-6a23c1539fe4\" (UID: \"4912afc9-a302-4518-88e2-6a23c1539fe4\") "
Nov 03 21:14:11.321864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.321813    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca" pod "4912afc9-a302-4518-88e2-6a23c1539fe4" (UID: "4912afc9-a302-4518-88e2-6a23c1539fe4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:11.322013 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.321998    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:11.323772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.323750    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca" pod "4912afc9-a302-4518-88e2-6a23c1539fe4" (UID: "4912afc9-a302-4518-88e2-6a23c1539fe4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:11.323904 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.323881    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj" pod "4912afc9-a302-4518-88e2-6a23c1539fe4" (UID: "4912afc9-a302-4518-88e2-6a23c1539fe4"). InnerVolumeSpecName "kube-api-access-5mzqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:11.423605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.423547    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4912afc9-a302-4518-88e2-6a23c1539fe4-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:11.423605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:11.423606    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mzqj\" (UniqueName: \"kubernetes.io/projected/4912afc9-a302-4518-88e2-6a23c1539fe4-kube-api-access-5mzqj\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:12.028048 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:12.028019    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"
Nov 03 21:14:12.028234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:12.028019    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx" event={"ID":"4912afc9-a302-4518-88e2-6a23c1539fe4","Type":"ContainerDied","Data":"2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f"}
Nov 03 21:14:12.028234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:12.028186    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2880a296b0072bb42396402de35ec05e3d8ea9742422c5ac626b71a3b6c7fa5f"
Nov 03 21:14:14.043425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.043385    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:14.481500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.481451    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main" containerID="cri-o://67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638" gracePeriod=45
Nov 03 21:14:14.529467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.529428    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main" containerID="cri-o://77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4" gracePeriod=45
Nov 03 21:14:14.795545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.795518    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:14:14.817135 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.817109    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:14:14.960864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.960824    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.960881    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.960930    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.960957    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.960991    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.961064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961061    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\" (UniqueName: \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.961328 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961109    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\" (UniqueName: \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961373    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.961411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961389    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.961518 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961447    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.961599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961527    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\" (UniqueName: \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.961653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961611    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.961714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961662    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\") pod \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\" (UID: \"8dd74d93-03e7-4d59-b5d2-53b539d55d03\") "
Nov 03 21:14:14.961775 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961760    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.961834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.961794    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\") pod \"b85e0994-00dc-40e4-9f7c-b54d13a34355\" (UID: \"b85e0994-00dc-40e4-9f7c-b54d13a34355\") "
Nov 03 21:14:14.962118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962075    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.962254 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962235    5915 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-home\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:14.962341 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962260    5915 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tmp-dir\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:14.962341 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962275    5915 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-model-cache\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:14.962341 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962288    5915 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-model-cache\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:14.962694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962668    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.962786 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.962744    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.964217 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.964191    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:14.964340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.964193    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:14.964742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.964711    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:14.965248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.965218    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "kube-api-access-vtssm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:14.965346 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.965322    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd" pod "8dd74d93-03e7-4d59-b5d2-53b539d55d03" (UID: "8dd74d93-03e7-4d59-b5d2-53b539d55d03"). InnerVolumeSpecName "kube-api-access-fkjtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:14.965499 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:14.965479    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm" pod "b85e0994-00dc-40e4-9f7c-b54d13a34355" (UID: "b85e0994-00dc-40e4-9f7c-b54d13a34355"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.051398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.051304    5915 generic.go:356] "Generic (PLEG): container finished" podID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerID="67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638" exitCode=0
Nov 03 21:14:15.051398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.051385    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" event={"ID":"b85e0994-00dc-40e4-9f7c-b54d13a34355","Type":"ContainerDied","Data":"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"}
Nov 03 21:14:15.052709 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.051419    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"
Nov 03 21:14:15.052709 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.051437    5915 scope.go:122] "RemoveContainer" containerID="67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"
Nov 03 21:14:15.054072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.054044    5915 generic.go:356] "Generic (PLEG): container finished" podID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerID="77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4" exitCode=0
Nov 03 21:14:15.054208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.054191    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"
Nov 03 21:14:15.054282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.051425    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg" event={"ID":"b85e0994-00dc-40e4-9f7c-b54d13a34355","Type":"ContainerDied","Data":"16423edf4f18ce9dfdabb1b2dbc27d138109395d1f7b71402881693b59d0ee3f"}
Nov 03 21:14:15.054352 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.054309    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" event={"ID":"8dd74d93-03e7-4d59-b5d2-53b539d55d03","Type":"ContainerDied","Data":"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"}
Nov 03 21:14:15.054352 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.054334    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl" event={"ID":"8dd74d93-03e7-4d59-b5d2-53b539d55d03","Type":"ContainerDied","Data":"f5bcee19b55fc3d8556678e38674f6586ae8ad4e9baa315776415cdab5cc475e"}
Nov 03 21:14:15.063122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063096    5915 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b85e0994-00dc-40e4-9f7c-b54d13a34355-tls-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063127    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkjtd\" (UniqueName: \"kubernetes.io/projected/8dd74d93-03e7-4d59-b5d2-53b539d55d03-kube-api-access-fkjtd\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063145    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtssm\" (UniqueName: \"kubernetes.io/projected/b85e0994-00dc-40e4-9f7c-b54d13a34355-kube-api-access-vtssm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063159    5915 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-tmp-dir\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063173    5915 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-home\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063185    5915 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b85e0994-00dc-40e4-9f7c-b54d13a34355-dshm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063198    5915 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dd74d93-03e7-4d59-b5d2-53b539d55d03-tls-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.063498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.063211    5915 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dd74d93-03e7-4d59-b5d2-53b539d55d03-dshm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.071294 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.071272    5915 scope.go:122] "RemoveContainer" containerID="67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"
Nov 03 21:14:15.071792 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:15.071759    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638\": container with ID starting with 67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638 not found: ID does not exist" containerID="67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"
Nov 03 21:14:15.071896 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.071807    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638"} err="failed to get container status \"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638\": rpc error: code = NotFound desc = could not find container \"67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638\": container with ID starting with 67fee3f569c7f7e50e98983d4d1f8e6e0f8d33650b078e0cf42e05955bcb4638 not found: ID does not exist"
Nov 03 21:14:15.071896 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.071835    5915 scope.go:122] "RemoveContainer" containerID="77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"
Nov 03 21:14:15.082172 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.082152    5915 scope.go:122] "RemoveContainer" containerID="77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"
Nov 03 21:14:15.082487 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:15.082461    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4\": container with ID starting with 77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4 not found: ID does not exist" containerID="77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"
Nov 03 21:14:15.082617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.082493    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4"} err="failed to get container status \"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4\": rpc error: code = NotFound desc = could not find container \"77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4\": container with ID starting with 77bdb9cbb0956dc0dd46595a708abdd524f1758c201d541f8abd94bf4940d8f4 not found: ID does not exist"
Nov 03 21:14:15.227767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.227733    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 21:14:15.230165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.230142    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-w6-12a423-kserve-57d67766fd-dmmpl"]
Nov 03 21:14:15.338732 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.338647    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:14:15.343601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.343574    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/distinct-2-w6-12a423-kserve-76c987ccbf-cwrcg"]
Nov 03 21:14:15.683255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.683163    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" path="/var/lib/kubelet/pods/8dd74d93-03e7-4d59-b5d2-53b539d55d03/volumes"
Nov 03 21:14:15.683856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:15.683833    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" path="/var/lib/kubelet/pods/b85e0994-00dc-40e4-9f7c-b54d13a34355/volumes"
Nov 03 21:14:18.641370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:18.641336    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:18.804526 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:18.804463    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:18.947908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:18.947837    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:23.392626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.392553    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 21:14:23.393090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.392798    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" podUID="89775bcb-6ffe-4af7-812e-1e7141bbca59" containerName="maas-api" containerID="cri-o://ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35" gracePeriod=30
Nov 03 21:14:23.496314 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.496270    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:23.701026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.700992    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 21:14:23.787244 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.784348    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\" (UniqueName: \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\") pod \"89775bcb-6ffe-4af7-812e-1e7141bbca59\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") "
Nov 03 21:14:23.787605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.787551    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") pod \"89775bcb-6ffe-4af7-812e-1e7141bbca59\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") "
Nov 03 21:14:23.787803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.787785    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\") pod \"89775bcb-6ffe-4af7-812e-1e7141bbca59\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") "
Nov 03 21:14:23.788168 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.788144    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\") pod \"89775bcb-6ffe-4af7-812e-1e7141bbca59\" (UID: \"89775bcb-6ffe-4af7-812e-1e7141bbca59\") "
Nov 03 21:14:23.791592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.791509    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca" pod "89775bcb-6ffe-4af7-812e-1e7141bbca59" (UID: "89775bcb-6ffe-4af7-812e-1e7141bbca59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:23.801052 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.799700    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t" pod "89775bcb-6ffe-4af7-812e-1e7141bbca59" (UID: "89775bcb-6ffe-4af7-812e-1e7141bbca59"). InnerVolumeSpecName "kube-api-access-rc22t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:23.806803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.806681    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls" pod "89775bcb-6ffe-4af7-812e-1e7141bbca59" (UID: "89775bcb-6ffe-4af7-812e-1e7141bbca59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:23.806803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.806781    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs" pod "89775bcb-6ffe-4af7-812e-1e7141bbca59" (UID: "89775bcb-6ffe-4af7-812e-1e7141bbca59"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:23.889415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.889381    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:23.889415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.889409    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc22t\" (UniqueName: \"kubernetes.io/projected/89775bcb-6ffe-4af7-812e-1e7141bbca59-kube-api-access-rc22t\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:23.889415 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.889422    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89775bcb-6ffe-4af7-812e-1e7141bbca59-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:23.889679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:23.889430    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89775bcb-6ffe-4af7-812e-1e7141bbca59-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:24.119552 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.119508    5915 generic.go:356] "Generic (PLEG): container finished" podID="89775bcb-6ffe-4af7-812e-1e7141bbca59" containerID="ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35" exitCode=0
Nov 03 21:14:24.119796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.119580    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"
Nov 03 21:14:24.119796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.119599    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" event={"ID":"89775bcb-6ffe-4af7-812e-1e7141bbca59","Type":"ContainerDied","Data":"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"}
Nov 03 21:14:24.119796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.119661    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht" event={"ID":"89775bcb-6ffe-4af7-812e-1e7141bbca59","Type":"ContainerDied","Data":"32ee4c1b5c9d17f58882cb4272a4a93ed51ef619902ccd11dd35d0b90390e60e"}
Nov 03 21:14:24.119796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.119693    5915 scope.go:122] "RemoveContainer" containerID="ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"
Nov 03 21:14:24.132412 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.132391    5915 scope.go:122] "RemoveContainer" containerID="ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"
Nov 03 21:14:24.132731 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:24.132709    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35\": container with ID starting with ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35 not found: ID does not exist" containerID="ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"
Nov 03 21:14:24.132789 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.132740    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35"} err="failed to get container status \"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35\": rpc error: code = NotFound desc = could not find container \"ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35\": container with ID starting with ec4d626a6c6b557ce12041e8a30574104a7a048ad75ae49d9ae9b1435be08c35 not found: ID does not exist"
Nov 03 21:14:24.288002 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.287971    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 21:14:24.341296 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:24.341245    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e22abf-7f577f4754-7l6ht"]
Nov 03 21:14:25.105801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:25.105763    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:25.355738 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:25.355701    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:25.680845 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:25.680809    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89775bcb-6ffe-4af7-812e-1e7141bbca59" path="/var/lib/kubelet/pods/89775bcb-6ffe-4af7-812e-1e7141bbca59/volumes"
Nov 03 21:14:25.976959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:25.976927    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:26.130929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.130895    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main" containerID="cri-o://7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe" gracePeriod=45
Nov 03 21:14:26.146698 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.146661    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" containerID="cri-o://11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2" gracePeriod=45
Nov 03 21:14:26.341595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.339295    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": dial tcp 10.132.0.70:8000: connect: connection refused"
Nov 03 21:14:26.458699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.458673    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:14:26.466873 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.466845    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:14:26.517398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517328    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\" (UniqueName: \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.517398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517374    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\" (UniqueName: \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517406    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\" (UniqueName: \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517447    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517464    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517507    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517541    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517652    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.517678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517678    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.518065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517727    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\") pod \"b9fee32f-9d31-455d-8b37-c302bf946472\" (UID: \"b9fee32f-9d31-455d-8b37-c302bf946472\") "
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.518992    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.519062    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\") pod \"593ad785-80ee-4929-966e-818906ba07c5\" (UID: \"593ad785-80ee-4929-966e-818906ba07c5\") "
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517744    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.517749    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.518376    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.518621    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.520102 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.520059    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.520681 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.520653    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:26.520791 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.520700    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.521735 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.521711    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.521876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.521777    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "kube-api-access-v2s6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:26.523303 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.523272    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:26.523860 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.523836    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm" pod "593ad785-80ee-4929-966e-818906ba07c5" (UID: "593ad785-80ee-4929-966e-818906ba07c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.524110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.524088    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69" pod "b9fee32f-9d31-455d-8b37-c302bf946472" (UID: "b9fee32f-9d31-455d-8b37-c302bf946472"). InnerVolumeSpecName "kube-api-access-q6k69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:26.621038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621002    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2s6c\" (UniqueName: \"kubernetes.io/projected/593ad785-80ee-4929-966e-818906ba07c5-kube-api-access-v2s6c\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621034    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6k69\" (UniqueName: \"kubernetes.io/projected/b9fee32f-9d31-455d-8b37-c302bf946472-kube-api-access-q6k69\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621044    5915 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/593ad785-80ee-4929-966e-818906ba07c5-tls-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621053    5915 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-dshm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621062    5915 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-model-cache\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621092    5915 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-tmp-dir\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621101    5915 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-dshm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621109    5915 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9fee32f-9d31-455d-8b37-c302bf946472-tls-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621117    5915 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-home\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621125    5915 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9fee32f-9d31-455d-8b37-c302bf946472-tmp-dir\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621132    5915 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-home\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.621333 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:26.621139    5915 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/593ad785-80ee-4929-966e-818906ba07c5-model-cache\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:27.141394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.141360    5915 generic.go:356] "Generic (PLEG): container finished" podID="593ad785-80ee-4929-966e-818906ba07c5" containerID="7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe" exitCode=0
Nov 03 21:14:27.141906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.141416    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"
Nov 03 21:14:27.141906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.141500    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" event={"ID":"593ad785-80ee-4929-966e-818906ba07c5","Type":"ContainerDied","Data":"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"}
Nov 03 21:14:27.141906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.141534    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz" event={"ID":"593ad785-80ee-4929-966e-818906ba07c5","Type":"ContainerDied","Data":"e8f3ea7330dd098267624d7113a37952d2adce7bac38ed70b7977cd5c56a79dd"}
Nov 03 21:14:27.141906 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.141588    5915 scope.go:122] "RemoveContainer" containerID="7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"
Nov 03 21:14:27.143588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.143459    5915 generic.go:356] "Generic (PLEG): container finished" podID="b9fee32f-9d31-455d-8b37-c302bf946472" containerID="11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2" exitCode=0
Nov 03 21:14:27.143588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.143514    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"
Nov 03 21:14:27.143588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.143514    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" event={"ID":"b9fee32f-9d31-455d-8b37-c302bf946472","Type":"ContainerDied","Data":"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"}
Nov 03 21:14:27.143856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.143603    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt" event={"ID":"b9fee32f-9d31-455d-8b37-c302bf946472","Type":"ContainerDied","Data":"bf4592ee46d15863accbf36c0ad65b62fe0cca1b26a4b512aba654c556611a5c"}
Nov 03 21:14:27.155272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.155084    5915 scope.go:122] "RemoveContainer" containerID="7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"
Nov 03 21:14:27.155416 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:27.155394    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe\": container with ID starting with 7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe not found: ID does not exist" containerID="7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"
Nov 03 21:14:27.155481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.155424    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe"} err="failed to get container status \"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe\": rpc error: code = NotFound desc = could not find container \"7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe\": container with ID starting with 7d3021ce4effbd3050131c0550b43d1592ce57cd91d80ecdc9e5700c77e64dbe not found: ID does not exist"
Nov 03 21:14:27.155481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.155442    5915 scope.go:122] "RemoveContainer" containerID="11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"
Nov 03 21:14:27.164683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.164663    5915 scope.go:122] "RemoveContainer" containerID="11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"
Nov 03 21:14:27.164939 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:27.164918    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2\": container with ID starting with 11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2 not found: ID does not exist" containerID="11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"
Nov 03 21:14:27.165011 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.164945    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2"} err="failed to get container status \"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2\": rpc error: code = NotFound desc = could not find container \"11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2\": container with ID starting with 11d2824c247394723e829fe021eede9519fb3f04c43c61a0c9b57931d29586c2 not found: ID does not exist"
Nov 03 21:14:27.308110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.308075    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 21:14:27.358728 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.358695    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-862138/simulator-w2-862138-kserve-5bfbf44957-zthkt"]
Nov 03 21:14:27.426249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.426170    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 21:14:27.431610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.431584    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-w2-862138-kserve-869bdc45bc-qrrbz"]
Nov 03 21:14:27.696644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.693929    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="593ad785-80ee-4929-966e-818906ba07c5" path="/var/lib/kubelet/pods/593ad785-80ee-4929-966e-818906ba07c5/volumes"
Nov 03 21:14:27.696644 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:27.694719    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" path="/var/lib/kubelet/pods/b9fee32f-9d31-455d-8b37-c302bf946472/volumes"
Nov 03 21:14:29.764988 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:29.764954    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 21:14:29.765407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:29.765147    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" podUID="512d6e6d-1e83-4c71-8715-ee44734cd867" containerName="maas-api" containerID="cri-o://190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71" gracePeriod=30
Nov 03 21:14:29.877933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:29.877883    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:30.032025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.031704    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:30.092144 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.092121    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 21:14:30.162578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.162492    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\") pod \"512d6e6d-1e83-4c71-8715-ee44734cd867\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") "
Nov 03 21:14:30.162578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.162551    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\") pod \"512d6e6d-1e83-4c71-8715-ee44734cd867\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") "
Nov 03 21:14:30.162836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.162610    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\" (UniqueName: \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\") pod \"512d6e6d-1e83-4c71-8715-ee44734cd867\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") "
Nov 03 21:14:30.162836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.162705    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\") pod \"512d6e6d-1e83-4c71-8715-ee44734cd867\" (UID: \"512d6e6d-1e83-4c71-8715-ee44734cd867\") "
Nov 03 21:14:30.163866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.163354    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca" pod "512d6e6d-1e83-4c71-8715-ee44734cd867" (UID: "512d6e6d-1e83-4c71-8715-ee44734cd867"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:30.163866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.163733    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/512d6e6d-1e83-4c71-8715-ee44734cd867-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:30.168546 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.168518    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls" pod "512d6e6d-1e83-4c71-8715-ee44734cd867" (UID: "512d6e6d-1e83-4c71-8715-ee44734cd867"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:30.168546 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.168527    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs" pod "512d6e6d-1e83-4c71-8715-ee44734cd867" (UID: "512d6e6d-1e83-4c71-8715-ee44734cd867"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:30.172717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.172688    5915 generic.go:356] "Generic (PLEG): container finished" podID="512d6e6d-1e83-4c71-8715-ee44734cd867" containerID="190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71" exitCode=0
Nov 03 21:14:30.172868 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.172775    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" event={"ID":"512d6e6d-1e83-4c71-8715-ee44734cd867","Type":"ContainerDied","Data":"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"}
Nov 03 21:14:30.172868 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.172835    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"
Nov 03 21:14:30.172868 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.172853    5915 scope.go:122] "RemoveContainer" containerID="190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"
Nov 03 21:14:30.173120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.172840    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq" event={"ID":"512d6e6d-1e83-4c71-8715-ee44734cd867","Type":"ContainerDied","Data":"a70fd7ad163e84cc3f189c80184e4204194effbc84396da82b11825b69dfce56"}
Nov 03 21:14:30.176420 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.175215    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x" pod "512d6e6d-1e83-4c71-8715-ee44734cd867" (UID: "512d6e6d-1e83-4c71-8715-ee44734cd867"). InnerVolumeSpecName "kube-api-access-xvg4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:30.190178 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.190161    5915 scope.go:122] "RemoveContainer" containerID="190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"
Nov 03 21:14:30.190481 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:30.190462    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71\": container with ID starting with 190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71 not found: ID does not exist" containerID="190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"
Nov 03 21:14:30.190524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.190491    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71"} err="failed to get container status \"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71\": rpc error: code = NotFound desc = could not find container \"190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71\": container with ID starting with 190a6003b0ca9b21bcec39fb07fc46a95827d4c1b539ed10be33193ba0cdfb71 not found: ID does not exist"
Nov 03 21:14:30.264319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.264286    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:30.264319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.264313    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvg4x\" (UniqueName: \"kubernetes.io/projected/512d6e6d-1e83-4c71-8715-ee44734cd867-kube-api-access-xvg4x\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:30.264319 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.264323    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/512d6e6d-1e83-4c71-8715-ee44734cd867-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:30.593291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.593258    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 21:14:30.605292 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:30.599405    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-862138-84bd488d55-kj8vq"]
Nov 03 21:14:31.027530 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:31.027481    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:31.398183 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:31.398100    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:31.672790 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:31.672714    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:31.680776 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:31.680751    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="512d6e6d-1e83-4c71-8715-ee44734cd867" path="/var/lib/kubelet/pods/512d6e6d-1e83-4c71-8715-ee44734cd867/volumes"
Nov 03 21:14:31.865537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:31.865505    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:32.538847 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:32.538808    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:32.727023 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:32.726993    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:32.970871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:32.970841    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:33.717548 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:33.717515    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:34.186257 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:34.186180    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:34.351081 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:34.351043    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:34.560025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:34.559995    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 21:14:34.560211 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:34.560187    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy" containerID="cri-o://8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e" gracePeriod=30
Nov 03 21:14:34.691925 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:34.691878    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:36.549104 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:36.549069    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:36.693829 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:36.693803    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:37.002649 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:37.002616    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:14:37.676337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:37.676305    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:38.293496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:38.293463    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:38.528348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:38.528316    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:38.776056 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:38.776027    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:39.102283 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.102202    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:39.804062 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.804040    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 21:14:39.866727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866662    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.866727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866706    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.866909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866736    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.866909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866794    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.866909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866832    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.866909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866867    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.867124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866913    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.867124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866938    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.867124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.866985    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.867124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867049    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\") pod \"c7670773-7dc2-42c1-872e-410ee798d9f7\" (UID: \"c7670773-7dc2-42c1-872e-410ee798d9f7\") "
Nov 03 21:14:39.867338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867204    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.867338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867275    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.867338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867316    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:39.867516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867491    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7670773-7dc2-42c1-872e-410ee798d9f7-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.867589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867452    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.867589 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867520    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.867679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867628    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.867679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.867644    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.869050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.869027    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "kube-api-access-r98n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:39.869363 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.869343    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.869679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.869662    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:39.869679 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.869665    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token" pod "c7670773-7dc2-42c1-872e-410ee798d9f7" (UID: "c7670773-7dc2-42c1-872e-410ee798d9f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:39.968540 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.968511    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.968540 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.968539    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r98n7\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-kube-api-access-r98n7\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.968756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.968549    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.968756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.968570    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.968756 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:39.968579    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7670773-7dc2-42c1-872e-410ee798d9f7-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.234373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.234340    5915 generic.go:356] "Generic (PLEG): container finished" podID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerID="8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e" exitCode=0
Nov 03 21:14:40.234574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.234383    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"
Nov 03 21:14:40.234574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.234414    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" event={"ID":"c7670773-7dc2-42c1-872e-410ee798d9f7","Type":"ContainerDied","Data":"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"}
Nov 03 21:14:40.234574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.234453    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp" event={"ID":"c7670773-7dc2-42c1-872e-410ee798d9f7","Type":"ContainerDied","Data":"75bd52bc4fd3d8b082629abf5f8b987ca86bea2d4baff4133960e1eeb4ac028f"}
Nov 03 21:14:40.234574 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.234474    5915 scope.go:122] "RemoveContainer" containerID="8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"
Nov 03 21:14:40.245015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.244996    5915 scope.go:122] "RemoveContainer" containerID="8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"
Nov 03 21:14:40.245299 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:40.245280    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e\": container with ID starting with 8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e not found: ID does not exist" containerID="8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"
Nov 03 21:14:40.245380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.245308    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e"} err="failed to get container status \"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e\": rpc error: code = NotFound desc = could not find container \"8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e\": container with ID starting with 8c4953875b10bea7792cf1e086124e314a5f9195b50a5e846795750ab5e65e7e not found: ID does not exist"
Nov 03 21:14:40.399122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.399088    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 21:14:40.447933 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.447898    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-v26tp"]
Nov 03 21:14:40.468586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.468547    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:40.620228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.620152    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:40.946641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.946538    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 21:14:40.947140 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.946791    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" containerID="cri-o://9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322" gracePeriod=30
Nov 03 21:14:40.973308 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:40.973276    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:41.680519 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:41.680486    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" path="/var/lib/kubelet/pods/c7670773-7dc2-42c1-872e-410ee798d9f7/volumes"
Nov 03 21:14:41.993989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:41.993960    5915 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:14:41.994367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:41.994009    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:14:42.884351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:42.884317    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:43.216221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:43.216191    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:43.733305 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:43.733276    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:43.981379 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:43.981342    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:44.133273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:44.133194    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:46.201968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.201945    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 21:14:46.260343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.260267    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:14:46.262476 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.262451    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:14:46.270897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.270872    5915 generic.go:356] "Generic (PLEG): container finished" podID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerID="9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322" exitCode=0
Nov 03 21:14:46.271007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.270936    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"
Nov 03 21:14:46.271007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.270941    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" event={"ID":"3c4c3db1-b042-4be3-9709-d056ca8f077e","Type":"ContainerDied","Data":"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"}
Nov 03 21:14:46.271007 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.270996    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr" event={"ID":"3c4c3db1-b042-4be3-9709-d056ca8f077e","Type":"ContainerDied","Data":"1b1933d51c502c0d6473f85290a520a814cd36ec0e585dd7da3b3ba4c81c23c6"}
Nov 03 21:14:46.271121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.271023    5915 scope.go:122] "RemoveContainer" containerID="9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"
Nov 03 21:14:46.275275 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.275254    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:14:46.277683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.277665    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:14:46.283060 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.283044    5915 scope.go:122] "RemoveContainer" containerID="9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"
Nov 03 21:14:46.283369 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:46.283348    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322\": container with ID starting with 9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322 not found: ID does not exist" containerID="9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"
Nov 03 21:14:46.283457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.283380    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322"} err="failed to get container status \"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322\": rpc error: code = NotFound desc = could not find container \"9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322\": container with ID starting with 9e8106ccc68f04c40c695d05b2bedf2eaa10e4e69eab7d6e1d83f0592eb67322 not found: ID does not exist"
Nov 03 21:14:46.305603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.305581    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 21:14:46.305785 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.305768    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-6778fc7d44-68lkl" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager" containerID="cri-o://a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82" gracePeriod=30
Nov 03 21:14:46.328777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.328756    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.328875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.328798    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.328929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.328873    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.328929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.328918    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.328994 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.328952    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.329047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329005    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.329047 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329032    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.329160 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329061    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.329160 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329145    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\") pod \"3c4c3db1-b042-4be3-9709-d056ca8f077e\" (UID: \"3c4c3db1-b042-4be3-9709-d056ca8f077e\") "
Nov 03 21:14:46.329272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329173    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.329272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329213    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.329272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329245    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.329452 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329409    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.329600 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329551    5915 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-data\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.329683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329608    5915 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.329683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329624    5915 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-workload-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.329683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329633    5915 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-credential-socket\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.329939 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.329910    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:46.331235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.331212    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:46.331378 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.331357    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "kube-api-access-f4pf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:46.331612 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.331595    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:46.331703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.331624    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy" pod "3c4c3db1-b042-4be3-9709-d056ca8f077e" (UID: "3c4c3db1-b042-4be3-9709-d056ca8f077e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.430958 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.430925    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4pf9\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-kube-api-access-f4pf9\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.430958 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.430958    5915 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c4c3db1-b042-4be3-9709-d056ca8f077e-istiod-ca-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.431138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.430972    5915 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-envoy\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.431138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.430985    5915 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-token\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.431138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.430997    5915 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c4c3db1-b042-4be3-9709-d056ca8f077e-istio-podinfo\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.560713 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.560689    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 21:14:46.633299 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.633271    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\") pod \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") "
Nov 03 21:14:46.633489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.633329    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\" (UniqueName: \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\") pod \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") "
Nov 03 21:14:46.633489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.633357    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\") pod \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\" (UID: \"eccae0f1-0296-4f10-a808-03c9f2bad8c7\") "
Nov 03 21:14:46.635599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.635546    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs" pod "eccae0f1-0296-4f10-a808-03c9f2bad8c7" (UID: "eccae0f1-0296-4f10-a808-03c9f2bad8c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:46.635599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.635549    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7" pod "eccae0f1-0296-4f10-a808-03c9f2bad8c7" (UID: "eccae0f1-0296-4f10-a808-03c9f2bad8c7"). InnerVolumeSpecName "kube-api-access-qt4b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:46.635739 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.635608    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert" pod "eccae0f1-0296-4f10-a808-03c9f2bad8c7" (UID: "eccae0f1-0296-4f10-a808-03c9f2bad8c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:46.657998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.657970    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:46.734924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.734897    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.734924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.734924    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt4b7\" (UniqueName: \"kubernetes.io/projected/eccae0f1-0296-4f10-a808-03c9f2bad8c7-kube-api-access-qt4b7\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.735125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.734936    5915 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eccae0f1-0296-4f10-a808-03c9f2bad8c7-webhook-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.799221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.799188    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 21:14:46.801425 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:46.801400    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-g84rr"]
Nov 03 21:14:47.278298 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.278218    5915 generic.go:356] "Generic (PLEG): container finished" podID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerID="a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82" exitCode=0
Nov 03 21:14:47.278298 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.278282    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-68lkl"
Nov 03 21:14:47.278827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.278296    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerDied","Data":"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"}
Nov 03 21:14:47.278827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.278351    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-68lkl" event={"ID":"eccae0f1-0296-4f10-a808-03c9f2bad8c7","Type":"ContainerDied","Data":"5e8a2a431968fe89c152d694dd672ffd35cbc70e961c83fadac8f2e7b76127d9"}
Nov 03 21:14:47.278827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.278383    5915 scope.go:122] "RemoveContainer" containerID="a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"
Nov 03 21:14:47.288405 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.288389    5915 scope.go:122] "RemoveContainer" containerID="f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"
Nov 03 21:14:47.296777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.296763    5915 scope.go:122] "RemoveContainer" containerID="a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"
Nov 03 21:14:47.296995 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:47.296978    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82\": container with ID starting with a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82 not found: ID does not exist" containerID="a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"
Nov 03 21:14:47.297031 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.297004    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82"} err="failed to get container status \"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82\": rpc error: code = NotFound desc = could not find container \"a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82\": container with ID starting with a39049c5978c8e0d374d9dbd887d1953b3f760a9821e67f63ba2442aac304f82 not found: ID does not exist"
Nov 03 21:14:47.297031 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.297021    5915 scope.go:122] "RemoveContainer" containerID="f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"
Nov 03 21:14:47.297251 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:14:47.297232    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494\": container with ID starting with f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494 not found: ID does not exist" containerID="f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"
Nov 03 21:14:47.297297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.297258    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494"} err="failed to get container status \"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494\": rpc error: code = NotFound desc = could not find container \"f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494\": container with ID starting with f1ee57da3561b1f4b7f0581ba78341947f1dfd9a4b72f851b0fea60d724a5494 not found: ID does not exist"
Nov 03 21:14:47.441653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.441622    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 21:14:47.491009 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.490914    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-68lkl"]
Nov 03 21:14:47.680074 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.680001    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" path="/var/lib/kubelet/pods/3c4c3db1-b042-4be3-9709-d056ca8f077e/volumes"
Nov 03 21:14:47.680466 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:47.680430    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" path="/var/lib/kubelet/pods/eccae0f1-0296-4f10-a808-03c9f2bad8c7/volumes"
Nov 03 21:14:48.322959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.322930    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:14:48.323358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323338    5915 cpu_manager.go:414] "containerMap: removing container" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:48.323358 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323352    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323369    5915 cpu_manager.go:414] "containerMap: removing container" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323376    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323387    5915 cpu_manager.go:414] "containerMap: removing container" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323395    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323406    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323410    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323423    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323428    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323438    5915 cpu_manager.go:414] "containerMap: removing container" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main"
Nov 03 21:14:48.323443 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323445    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323458    5915 cpu_manager.go:414] "containerMap: removing container" podUID="512d6e6d-1e83-4c71-8715-ee44734cd867" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323463    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="512d6e6d-1e83-4c71-8715-ee44734cd867" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323475    5915 cpu_manager.go:414] "containerMap: removing container" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323479    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323486    5915 cpu_manager.go:414] "containerMap: removing container" podUID="89775bcb-6ffe-4af7-812e-1e7141bbca59" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323490    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89775bcb-6ffe-4af7-812e-1e7141bbca59" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323496    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4912afc9-a302-4518-88e2-6a23c1539fe4" containerName="revoke-keys"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323500    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4912afc9-a302-4518-88e2-6a23c1539fe4" containerName="revoke-keys"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323512    5915 cpu_manager.go:414] "containerMap: removing container" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323519    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323530    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323534    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323618    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9fee32f-9d31-455d-8b37-c302bf946472" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323627    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="593ad785-80ee-4929-966e-818906ba07c5" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323633    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="512d6e6d-1e83-4c71-8715-ee44734cd867" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323640    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323644    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="89775bcb-6ffe-4af7-812e-1e7141bbca59" containerName="maas-api"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323649    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4912afc9-a302-4518-88e2-6a23c1539fe4" containerName="revoke-keys"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323655    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad9a34e6-a541-4cba-b714-a76243c87f38" containerName="istio-proxy"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323662    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dd74d93-03e7-4d59-b5d2-53b539d55d03" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323666    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b85e0994-00dc-40e4-9f7c-b54d13a34355" containerName="main"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323672    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7670773-7dc2-42c1-872e-410ee798d9f7" containerName="istio-proxy"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323678    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="eccae0f1-0296-4f10-a808-03c9f2bad8c7" containerName="manager"
Nov 03 21:14:48.323757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.323686    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c4c3db1-b042-4be3-9709-d056ca8f077e" containerName="istio-proxy"
Nov 03 21:14:48.328182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.328165    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.429795 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.429762    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 03 21:14:48.429795 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.429770    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tgsmh\""
Nov 03 21:14:48.430035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.429842    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 03 21:14:48.430035 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.429885    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 03 21:14:48.432452 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.432430    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:14:48.451790 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.451765    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8289r\" (UniqueName: \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.451909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.451822    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.452022 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.452003    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.553199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.553168    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.553199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.553202    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8289r\" (UniqueName: \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.553393 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.553235    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.556134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.556111    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.556278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.556191    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.614484 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.614405    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8289r\" (UniqueName: \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\") pod \"maas-controller-6778fc7d44-k9tvw\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") " pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:48.642388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:48.642364    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:49.024466 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:49.024438    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:14:49.026819 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:14:49.026793    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2de86ae_04dc_48ef_a20c_509f0bc80a45.slice/crio-5f0c611f80004140802a9a74d5e00f4a0e866b095627bef35ec345c6d98086eb WatchSource:0}: Error finding container 5f0c611f80004140802a9a74d5e00f4a0e866b095627bef35ec345c6d98086eb: Status 404 returned error can't find the container with id 5f0c611f80004140802a9a74d5e00f4a0e866b095627bef35ec345c6d98086eb
Nov 03 21:14:49.028190 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:49.028171    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:14:49.295176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:49.295103    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" event={"ID":"f2de86ae-04dc-48ef-a20c-509f0bc80a45","Type":"ContainerStarted","Data":"5f0c611f80004140802a9a74d5e00f4a0e866b095627bef35ec345c6d98086eb"}
Nov 03 21:14:50.303852 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:50.303815    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" event={"ID":"f2de86ae-04dc-48ef-a20c-509f0bc80a45","Type":"ContainerStarted","Data":"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"}
Nov 03 21:14:50.304282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:50.303913    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:14:50.422213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:50.422154    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" podStartSLOduration=1.560023583 podStartE2EDuration="2.422136322s" totalImagesPullingTime="862.112739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:14:50.420884786 +0000 UTC m=+2104.872253967" watchObservedRunningTime="2026-11-03 21:14:50.422136322 +0000 UTC m=+2104.873505478"
Nov 03 21:14:53.636955 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:53.636923    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:55.346879 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:55.346845    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:14:59.633726 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:14:59.633689    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:00.168079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.168042    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:00.173608 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.173591    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.227701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.227657    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:15:00.227701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.227687    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:15:00.260329 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.260292    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.260494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.260361    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gct5v\" (UniqueName: \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.277709 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.277687    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:15:00.279858 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.279839    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:00.360935 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.360908    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.361135 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.360968    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gct5v\" (UniqueName: \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.361713 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.361684    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.422586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.422479    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gct5v\" (UniqueName: \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\") pod \"maas-api-key-cleanup-29895675-nbbcw\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.485835 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.485800    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:00.857457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:00.857429    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:00.859288 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:15:00.859248    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf052374a_8da4_496c_87f8_cf24c23affe0.slice/crio-1f4869abc5a178fc22f6363cb933d6cace3c00f94b56ed223d527b9f4e1f7129 WatchSource:0}: Error finding container 1f4869abc5a178fc22f6363cb933d6cace3c00f94b56ed223d527b9f4e1f7129: Status 404 returned error can't find the container with id 1f4869abc5a178fc22f6363cb933d6cace3c00f94b56ed223d527b9f4e1f7129
Nov 03 21:15:01.314962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:01.314937    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:15:01.370428 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:01.370342    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerStarted","Data":"a5dbf895aa8673868b2a88599b75d3a5f7de7b9e0549a1ca8ddf11aaf498ba4a"}
Nov 03 21:15:01.370428 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:01.370392    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerStarted","Data":"1f4869abc5a178fc22f6363cb933d6cace3c00f94b56ed223d527b9f4e1f7129"}
Nov 03 21:15:01.558408 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:01.558358    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" podStartSLOduration=1.558343694 podStartE2EDuration="1.558343694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:15:01.557732695 +0000 UTC m=+2116.009101871" watchObservedRunningTime="2026-11-03 21:15:01.558343694 +0000 UTC m=+2116.009712870"
Nov 03 21:15:01.729597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:01.729521    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:02.377766 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:02.377725    5915 generic.go:356] "Generic (PLEG): container finished" podID="f052374a-8da4-496c-87f8-cf24c23affe0" containerID="a5dbf895aa8673868b2a88599b75d3a5f7de7b9e0549a1ca8ddf11aaf498ba4a" exitCode=60
Nov 03 21:15:02.378214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:02.377813    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerDied","Data":"a5dbf895aa8673868b2a88599b75d3a5f7de7b9e0549a1ca8ddf11aaf498ba4a"}
Nov 03 21:15:02.378214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:02.378030    5915 scope.go:122] "RemoveContainer" containerID="a5dbf895aa8673868b2a88599b75d3a5f7de7b9e0549a1ca8ddf11aaf498ba4a"
Nov 03 21:15:03.386500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:03.386465    5915 generic.go:356] "Generic (PLEG): container finished" podID="f052374a-8da4-496c-87f8-cf24c23affe0" containerID="e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede" exitCode=60
Nov 03 21:15:03.386996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:03.386591    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerDied","Data":"e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede"}
Nov 03 21:15:03.386996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:03.386646    5915 scope.go:122] "RemoveContainer" containerID="a5dbf895aa8673868b2a88599b75d3a5f7de7b9e0549a1ca8ddf11aaf498ba4a"
Nov 03 21:15:03.386996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:03.386934    5915 scope.go:122] "RemoveContainer" containerID="e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede"
Nov 03 21:15:03.387242 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:03.387221    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895675-nbbcw_odh-ai-gateway-infra(f052374a-8da4-496c-87f8-cf24c23affe0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" podUID="f052374a-8da4-496c-87f8-cf24c23affe0"
Nov 03 21:15:04.394186 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:04.394156    5915 scope.go:122] "RemoveContainer" containerID="e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede"
Nov 03 21:15:04.394708 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:04.394431    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895675-nbbcw_odh-ai-gateway-infra(f052374a-8da4-496c-87f8-cf24c23affe0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" podUID="f052374a-8da4-496c-87f8-cf24c23affe0"
Nov 03 21:15:07.009761 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:07.009688    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:15:07.086623 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:07.086534    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:11.602463 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:11.602432    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:11.779454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:11.779422    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:17.673754 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:17.673725    5915 scope.go:122] "RemoveContainer" containerID="e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede"
Nov 03 21:15:18.517506 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:18.517479    5915 generic.go:356] "Generic (PLEG): container finished" podID="f052374a-8da4-496c-87f8-cf24c23affe0" containerID="9a74c8e6fa560ce17f8e5a8c0b7f991740b034c23883183c81139fd9fe0eea19" exitCode=60
Nov 03 21:15:18.517653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:18.517570    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerDied","Data":"9a74c8e6fa560ce17f8e5a8c0b7f991740b034c23883183c81139fd9fe0eea19"}
Nov 03 21:15:18.517653 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:18.517636    5915 scope.go:122] "RemoveContainer" containerID="e2397991c186b62da399cf4c7595e12de35d8fa74c46fad39af01bda86648ede"
Nov 03 21:15:18.517962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:18.517941    5915 scope.go:122] "RemoveContainer" containerID="9a74c8e6fa560ce17f8e5a8c0b7f991740b034c23883183c81139fd9fe0eea19"
Nov 03 21:15:18.518243 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:18.518220    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895675-nbbcw_odh-ai-gateway-infra(f052374a-8da4-496c-87f8-cf24c23affe0)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" podUID="f052374a-8da4-496c-87f8-cf24c23affe0"
Nov 03 21:15:19.295361 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.295330    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:19.664711 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.664631    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:19.802842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.802818    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:19.891812 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.891787    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\") pod \"f052374a-8da4-496c-87f8-cf24c23affe0\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") "
Nov 03 21:15:19.891946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.891866    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\" (UniqueName: \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\") pod \"f052374a-8da4-496c-87f8-cf24c23affe0\" (UID: \"f052374a-8da4-496c-87f8-cf24c23affe0\") "
Nov 03 21:15:19.892188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.892157    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca" pod "f052374a-8da4-496c-87f8-cf24c23affe0" (UID: "f052374a-8da4-496c-87f8-cf24c23affe0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:15:19.894078 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.894059    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v" pod "f052374a-8da4-496c-87f8-cf24c23affe0" (UID: "f052374a-8da4-496c-87f8-cf24c23affe0"). InnerVolumeSpecName "kube-api-access-gct5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:15:19.993267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.993233    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f052374a-8da4-496c-87f8-cf24c23affe0-maas-api-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.993431 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:19.993272    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gct5v\" (UniqueName: \"kubernetes.io/projected/f052374a-8da4-496c-87f8-cf24c23affe0-kube-api-access-gct5v\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:20.532894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:20.532863    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"
Nov 03 21:15:20.533334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:20.532904    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw" event={"ID":"f052374a-8da4-496c-87f8-cf24c23affe0","Type":"ContainerDied","Data":"1f4869abc5a178fc22f6363cb933d6cace3c00f94b56ed223d527b9f4e1f7129"}
Nov 03 21:15:20.533334 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:20.532980    5915 scope.go:122] "RemoveContainer" containerID="9a74c8e6fa560ce17f8e5a8c0b7f991740b034c23883183c81139fd9fe0eea19"
Nov 03 21:15:20.710477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:20.710445    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:20.759768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:20.759741    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895675-nbbcw"]
Nov 03 21:15:21.680259 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:21.680228    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" path="/var/lib/kubelet/pods/f052374a-8da4-496c-87f8-cf24c23affe0/volumes"
Nov 03 21:15:23.824521 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:23.824492    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:23.940546 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:23.940512    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:15:23.940763 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:23.940723    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" podUID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" containerName="manager" containerID="cri-o://34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e" gracePeriod=30
Nov 03 21:15:24.212625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.212600    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:15:24.337605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.337548    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\") pod \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") "
Nov 03 21:15:24.337799 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.337665    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\" (UniqueName: \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\") pod \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") "
Nov 03 21:15:24.337799 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.337787    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\") pod \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\" (UID: \"f2de86ae-04dc-48ef-a20c-509f0bc80a45\") "
Nov 03 21:15:24.340606 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.340542    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert" pod "f2de86ae-04dc-48ef-a20c-509f0bc80a45" (UID: "f2de86ae-04dc-48ef-a20c-509f0bc80a45"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:15:24.341547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.341518    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs" pod "f2de86ae-04dc-48ef-a20c-509f0bc80a45" (UID: "f2de86ae-04dc-48ef-a20c-509f0bc80a45"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:15:24.341715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.341696    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r" pod "f2de86ae-04dc-48ef-a20c-509f0bc80a45" (UID: "f2de86ae-04dc-48ef-a20c-509f0bc80a45"). InnerVolumeSpecName "kube-api-access-8289r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:15:24.439125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.439034    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:24.439125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.439061    5915 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2de86ae-04dc-48ef-a20c-509f0bc80a45-webhook-cert\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:24.439125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.439071    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8289r\" (UniqueName: \"kubernetes.io/projected/f2de86ae-04dc-48ef-a20c-509f0bc80a45-kube-api-access-8289r\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:24.556755 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.556723    5915 generic.go:356] "Generic (PLEG): container finished" podID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" containerID="34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e" exitCode=0
Nov 03 21:15:24.556938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.556776    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-k9tvw"
Nov 03 21:15:24.556938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.556807    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" event={"ID":"f2de86ae-04dc-48ef-a20c-509f0bc80a45","Type":"ContainerDied","Data":"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"}
Nov 03 21:15:24.556938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.556858    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-k9tvw" event={"ID":"f2de86ae-04dc-48ef-a20c-509f0bc80a45","Type":"ContainerDied","Data":"5f0c611f80004140802a9a74d5e00f4a0e866b095627bef35ec345c6d98086eb"}
Nov 03 21:15:24.556938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.556880    5915 scope.go:122] "RemoveContainer" containerID="34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"
Nov 03 21:15:24.567093 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.567076    5915 scope.go:122] "RemoveContainer" containerID="34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"
Nov 03 21:15:24.567321 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:24.567304    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e\": container with ID starting with 34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e not found: ID does not exist" containerID="34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"
Nov 03 21:15:24.567368 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.567329    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e"} err="failed to get container status \"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e\": rpc error: code = NotFound desc = could not find container \"34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e\": container with ID starting with 34af2fe8560a72c281e60a6e86da6222f116b01aa4049e942993d05e1818b56e not found: ID does not exist"
Nov 03 21:15:24.719917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.719884    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:15:24.769338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:24.769241    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-k9tvw"]
Nov 03 21:15:25.680106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:25.680074    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" path="/var/lib/kubelet/pods/f2de86ae-04dc-48ef-a20c-509f0bc80a45/volumes"
Nov 03 21:15:26.042355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.042308    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:26.212140 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212109    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6778fc7d44-hlt4w"]
Nov 03 21:15:26.212544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212532    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212545    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212568    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212573    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212589    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" containerName="manager"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212593    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" containerName="manager"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212601    5915 cpu_manager.go:414] "containerMap: removing container" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212606    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212695    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212709    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.212869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212720    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2de86ae-04dc-48ef-a20c-509f0bc80a45" containerName="manager"
Nov 03 21:15:26.212869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.212726    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="f052374a-8da4-496c-87f8-cf24c23affe0" containerName="cleanup"
Nov 03 21:15:26.217154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.217138    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.318656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.318549    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 03 21:15:26.318656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.318584    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 03 21:15:26.318656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.318649    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 03 21:15:26.318894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.318661    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tgsmh\""
Nov 03 21:15:26.321235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.321215    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-hlt4w"]
Nov 03 21:15:26.356392 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.356360    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-webhook-cert\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.356547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.356407    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdwx\" (UniqueName: \"kubernetes.io/projected/30165c7a-d42c-464b-81b0-800805bb5f3d-kube-api-access-zcdwx\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.356547 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.356515    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-metrics-certs\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.457282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.457253    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdwx\" (UniqueName: \"kubernetes.io/projected/30165c7a-d42c-464b-81b0-800805bb5f3d-kube-api-access-zcdwx\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.457427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.457310    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-metrics-certs\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.457427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.457365    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-webhook-cert\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.460012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.459980    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-metrics-certs\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.460118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.460011    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/30165c7a-d42c-464b-81b0-800805bb5f3d-webhook-cert\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.517947 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.517926    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdwx\" (UniqueName: \"kubernetes.io/projected/30165c7a-d42c-464b-81b0-800805bb5f3d-kube-api-access-zcdwx\") pod \"maas-controller-6778fc7d44-hlt4w\" (UID: \"30165c7a-d42c-464b-81b0-800805bb5f3d\") " pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.529784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.529762    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:26.906920 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:26.906893    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6778fc7d44-hlt4w"]
Nov 03 21:15:26.908852 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:15:26.908822    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30165c7a_d42c_464b_81b0_800805bb5f3d.slice/crio-e9afe79c4e7bb7f9ab4aac018897f118aa18b3bc95f0e79352adb2f46cd217e5 WatchSource:0}: Error finding container e9afe79c4e7bb7f9ab4aac018897f118aa18b3bc95f0e79352adb2f46cd217e5: Status 404 returned error can't find the container with id e9afe79c4e7bb7f9ab4aac018897f118aa18b3bc95f0e79352adb2f46cd217e5
Nov 03 21:15:27.578316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.578279    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-hlt4w" event={"ID":"30165c7a-d42c-464b-81b0-800805bb5f3d","Type":"ContainerStarted","Data":"e9afe79c4e7bb7f9ab4aac018897f118aa18b3bc95f0e79352adb2f46cd217e5"}
Nov 03 21:15:27.818398 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.818361    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"]
Nov 03 21:15:27.821984 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.821968    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871602    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871639    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871656    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871774 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871726    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db44974-2a96-495d-bccd-290898b0c2af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871951 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871848    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtbft\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-kube-api-access-xtbft\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.871996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871970    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db44974-2a96-495d-bccd-290898b0c2af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.872041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.871999    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.872041 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.872024    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.872173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.872158    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973317 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973285    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db44974-2a96-495d-bccd-290898b0c2af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973324    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973353    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973396    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973422    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973458    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.973784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973625    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973848    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973878    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973908    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974295 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.973879    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db44974-2a96-495d-bccd-290898b0c2af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974295 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.974236    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtbft\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-kube-api-access-xtbft\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.974598    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1db44974-2a96-495d-bccd-290898b0c2af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.974742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.974622    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.976054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.976036    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1db44974-2a96-495d-bccd-290898b0c2af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:27.976457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:27.976441    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1db44974-2a96-495d-bccd-290898b0c2af-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:28.029851 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.029822    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"]
Nov 03 21:15:28.036188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.036152    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:28.036315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.036287    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtbft\" (UniqueName: \"kubernetes.io/projected/1db44974-2a96-495d-bccd-290898b0c2af-kube-api-access-xtbft\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8\" (UID: \"1db44974-2a96-495d-bccd-290898b0c2af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:28.135036 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.134950    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:28.561978 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.561955    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"]
Nov 03 21:15:28.564032 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:15:28.564003    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db44974_2a96_495d_bccd_290898b0c2af.slice/crio-d4437f07f75adee9378dcadac6d99e95729885749bac29896ec5ba4baa9dd593 WatchSource:0}: Error finding container d4437f07f75adee9378dcadac6d99e95729885749bac29896ec5ba4baa9dd593: Status 404 returned error can't find the container with id d4437f07f75adee9378dcadac6d99e95729885749bac29896ec5ba4baa9dd593
Nov 03 21:15:28.566280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.566245    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:15:28.566384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.566313    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:15:28.566384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.566341    5915 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:15:28.585989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.585957    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6778fc7d44-hlt4w" event={"ID":"30165c7a-d42c-464b-81b0-800805bb5f3d","Type":"ContainerStarted","Data":"8c670fe548658df5a892438c4dea60af687a4fd7d7a5518f38fa088a6d38be7f"}
Nov 03 21:15:28.586105 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.586025    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:28.587451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.587430    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" event={"ID":"1db44974-2a96-495d-bccd-290898b0c2af","Type":"ContainerStarted","Data":"d4437f07f75adee9378dcadac6d99e95729885749bac29896ec5ba4baa9dd593"}
Nov 03 21:15:28.705692 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:28.705629    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-6778fc7d44-hlt4w" podStartSLOduration=1.931429845 podStartE2EDuration="2.705607949s" totalImagesPullingTime="774.178104ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:15:28.702948729 +0000 UTC m=+2143.154317905" watchObservedRunningTime="2026-11-03 21:15:28.705607949 +0000 UTC m=+2143.156977127"
Nov 03 21:15:29.595492 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:29.595450    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" event={"ID":"1db44974-2a96-495d-bccd-290898b0c2af","Type":"ContainerStarted","Data":"ed873c76506b458e5412919dd1377cd5e6e68de9ae018b5f46089991ddaa61dc"}
Nov 03 21:15:29.765894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:29.765832    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podStartSLOduration=2.765819431 podStartE2EDuration="2.765819431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:15:29.763268444 +0000 UTC m=+2144.214637638" watchObservedRunningTime="2026-11-03 21:15:29.765819431 +0000 UTC m=+2144.217188608"
Nov 03 21:15:30.135727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:30.135694    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:30.137281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:30.137255    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:30.137427 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:30.137308    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:31.136370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:31.136332    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:31.136796 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:31.136391    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:32.135811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:32.135780    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:32.135993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:32.135849    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:33.135876 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:33.135835    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:33.136277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:33.135909    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:33.239345 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:33.239300    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:33.389919 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:33.389826    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:15:34.135424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:34.135307    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:34.135424 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:34.135369    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:35.135544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:35.135508    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused" start-of-body=
Nov 03 21:15:35.135784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:35.135594    5915 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": dial tcp 10.132.0.149:15021: connect: connection refused"
Nov 03 21:15:36.795136 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:36.795105    5915 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:36.795667 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:36.795513    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:36.796297 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:36.796276    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8"
Nov 03 21:15:37.002655 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:15:37.002623    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:15:39.598702 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:39.598672    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6778fc7d44-hlt4w"
Nov 03 21:15:43.562155 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:15:43.562119    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:01.244238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:01.244156    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:01.391091 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:01.391062    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:07.003653 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:16:07.003604    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:16:17.911196 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.911164    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"]
Nov 03 21:16:17.914822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.914804    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:17.967270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.967241    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:16:17.970773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.970745    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:17.970911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.970807    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:17.971059 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:17.971037    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9rhp\" (UniqueName: \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.016454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.016422    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:16:18.016454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.016437    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:16:18.016454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.016445    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:16:18.019111 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.019094    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"]
Nov 03 21:16:18.072004 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.071980    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9rhp\" (UniqueName: \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.072134 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.072038    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.072235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.072210    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.072838 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.072820    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.074339 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.074322    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.132664 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.132642    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9rhp\" (UniqueName: \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\") pod \"maas-api-revoke-sub-c1e271a9-j54gn\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.228886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.228847    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:18.617765 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.617741    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"]
Nov 03 21:16:18.619623 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:16:18.619595    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efcff1e_8fdc_4cfe_91c1_f0ece90a9a2e.slice/crio-2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341 WatchSource:0}: Error finding container 2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341: Status 404 returned error can't find the container with id 2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341
Nov 03 21:16:18.935447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.935413    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn" event={"ID":"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e","Type":"ContainerStarted","Data":"6f213688c30f261f68ecab680f0fef337e36dd5e2d6e1690cd07ab9e4acdd1d8"}
Nov 03 21:16:18.935883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:18.935470    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn" event={"ID":"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e","Type":"ContainerStarted","Data":"2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341"}
Nov 03 21:16:19.051039 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:19.050981    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn" podStartSLOduration=2.050966013 podStartE2EDuration="2.050966013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:16:19.049749576 +0000 UTC m=+2193.501118754" watchObservedRunningTime="2026-11-03 21:16:19.050966013 +0000 UTC m=+2193.502335184"
Nov 03 21:16:19.637120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:19.637090    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:19.943347 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:19.943250    5915 generic.go:356] "Generic (PLEG): container finished" podID="7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" containerID="6f213688c30f261f68ecab680f0fef337e36dd5e2d6e1690cd07ab9e4acdd1d8" exitCode=0
Nov 03 21:16:19.943822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:19.943337    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn" event={"ID":"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e","Type":"ContainerDied","Data":"6f213688c30f261f68ecab680f0fef337e36dd5e2d6e1690cd07ab9e4acdd1d8"}
Nov 03 21:16:21.085489 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.085466    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:21.098720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.098697    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\") pod \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") "
Nov 03 21:16:21.098836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.098753    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\") pod \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") "
Nov 03 21:16:21.098836 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.098804    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\" (UniqueName: \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\") pod \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\" (UID: \"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e\") "
Nov 03 21:16:21.099036 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.099007    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca" pod "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" (UID: "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:16:21.100895 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.100874    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca" pod "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" (UID: "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:16:21.101028 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.101010    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp" pod "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" (UID: "7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e"). InnerVolumeSpecName "kube-api-access-n9rhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:16:21.199980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.199957    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:21.199980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.199980    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9rhp\" (UniqueName: \"kubernetes.io/projected/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-kube-api-access-n9rhp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:21.200139 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.199992    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:21.956616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.956592    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"
Nov 03 21:16:21.956777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.956624    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn" event={"ID":"7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e","Type":"ContainerDied","Data":"2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341"}
Nov 03 21:16:21.956777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:21.956670    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ec1d70c3a2261e7f06cbecf88d11cc59c8142277aacb6aeb2b0c62ab52a5341"
Nov 03 21:16:37.019819 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:16:37.019772    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:16:42.775034 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:42.774999    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:42.938676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:42.938646    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:43.904153 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:43.904120    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:44.403822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:44.403788    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:16:49.439587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.439526    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"]
Nov 03 21:16:49.440180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.440158    5915 cpu_manager.go:414] "containerMap: removing container" podUID="7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" containerName="revoke-keys"
Nov 03 21:16:49.440279 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.440181    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" containerName="revoke-keys"
Nov 03 21:16:49.440345 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.440332    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" containerName="revoke-keys"
Nov 03 21:16:49.443657 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.443635    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.495758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.495731    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:16:49.495758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.495738    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:16:49.548727 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.548693    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"]
Nov 03 21:16:49.595775 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.595747    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:16:49.595944 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.595915    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:16:49.612686 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.612659    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5krql\" (UniqueName: \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.612813 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.612713    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.612813 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.612742    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.713782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.713753    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5krql\" (UniqueName: \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.713929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.713832    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.713929 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.713860    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.714671 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.714649    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.716510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.716487    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:49.774388 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:49.774357    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5krql\" (UniqueName: \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\") pod \"maas-api-revoke-sub-2e1f5b5f-vk94f\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:50.057198 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:50.057124    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:50.430082 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:50.430057    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"]
Nov 03 21:16:50.432492 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:16:50.432463    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4731beef_8a2d_47b2_9235_76e3f6b2c1f4.slice/crio-b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9 WatchSource:0}: Error finding container b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9: Status 404 returned error can't find the container with id b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9
Nov 03 21:16:51.153695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:51.153657    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f" event={"ID":"4731beef-8a2d-47b2-9235-76e3f6b2c1f4","Type":"ContainerStarted","Data":"141f4fcf8079d63103c5a5a295b589dc12f24bed79e659cd7ce8a029f900fb0f"}
Nov 03 21:16:51.154082 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:51.153708    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f" event={"ID":"4731beef-8a2d-47b2-9235-76e3f6b2c1f4","Type":"ContainerStarted","Data":"b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9"}
Nov 03 21:16:51.271378 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:51.271328    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f" podStartSLOduration=2.2713120399999998 podStartE2EDuration="2.27131204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:16:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:16:51.26957049 +0000 UTC m=+2225.720939659" watchObservedRunningTime="2026-11-03 21:16:51.27131204 +0000 UTC m=+2225.722681214"
Nov 03 21:16:52.161088 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:52.161053    5915 generic.go:356] "Generic (PLEG): container finished" podID="4731beef-8a2d-47b2-9235-76e3f6b2c1f4" containerID="141f4fcf8079d63103c5a5a295b589dc12f24bed79e659cd7ce8a029f900fb0f" exitCode=0
Nov 03 21:16:52.161597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:52.161108    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f" event={"ID":"4731beef-8a2d-47b2-9235-76e3f6b2c1f4","Type":"ContainerDied","Data":"141f4fcf8079d63103c5a5a295b589dc12f24bed79e659cd7ce8a029f900fb0f"}
Nov 03 21:16:53.319957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.319930    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:53.348467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.348440    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\" (UniqueName: \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\") pod \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") "
Nov 03 21:16:53.348603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.348540    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\") pod \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") "
Nov 03 21:16:53.350635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.350611    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca" pod "4731beef-8a2d-47b2-9235-76e3f6b2c1f4" (UID: "4731beef-8a2d-47b2-9235-76e3f6b2c1f4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:16:53.350716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.350679    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql" pod "4731beef-8a2d-47b2-9235-76e3f6b2c1f4" (UID: "4731beef-8a2d-47b2-9235-76e3f6b2c1f4"). InnerVolumeSpecName "kube-api-access-5krql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:16:53.449239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.449207    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\") pod \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\" (UID: \"4731beef-8a2d-47b2-9235-76e3f6b2c1f4\") "
Nov 03 21:16:53.449458 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.449443    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5krql\" (UniqueName: \"kubernetes.io/projected/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-kube-api-access-5krql\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:53.449545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.449460    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:53.449545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.449509    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca" pod "4731beef-8a2d-47b2-9235-76e3f6b2c1f4" (UID: "4731beef-8a2d-47b2-9235-76e3f6b2c1f4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:16:53.550030 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:53.550000    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4731beef-8a2d-47b2-9235-76e3f6b2c1f4-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:16:54.173421 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:54.173394    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"
Nov 03 21:16:54.173637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:54.173442    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f" event={"ID":"4731beef-8a2d-47b2-9235-76e3f6b2c1f4","Type":"ContainerDied","Data":"b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9"}
Nov 03 21:16:54.173637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:54.173491    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3cb7171e658219a8971b1b4ae80435e06a4745c5f95fb18b0f11efade9baba9"
Nov 03 21:16:58.703218 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:16:58.703188    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:01.970447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:01.970405    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"]
Nov 03 21:17:01.970970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:01.970946    5915 cpu_manager.go:414] "containerMap: removing container" podUID="4731beef-8a2d-47b2-9235-76e3f6b2c1f4" containerName="revoke-keys"
Nov 03 21:17:01.970970 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:01.970969    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4731beef-8a2d-47b2-9235-76e3f6b2c1f4" containerName="revoke-keys"
Nov 03 21:17:01.971310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:01.971100    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="4731beef-8a2d-47b2-9235-76e3f6b2c1f4" containerName="revoke-keys"
Nov 03 21:17:01.975143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:01.975122    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.025772 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.025736    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.025923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.025782    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.025923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.025866    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj6sk\" (UniqueName: \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.079614 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.079552    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:17:02.079614 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.079593    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:17:02.079830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.079687    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:17:02.079830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.079687    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:17:02.082282 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.082262    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"]
Nov 03 21:17:02.127238 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.127209    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.127382 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.127247    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.127382 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.127275    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj6sk\" (UniqueName: \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.127912 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.127891    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.129751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.129728    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.187927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.187895    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj6sk\" (UniqueName: \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\") pod \"maas-api-revoke-sub-ec843018-4z4z9\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.290089 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.290023    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:02.670084 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:02.669986    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"]
Nov 03 21:17:03.226685 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:03.226649    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9" event={"ID":"533409ce-58a9-4dd3-9767-666c2b90684b","Type":"ContainerStarted","Data":"3f4accf669731367ecd26d1db64ef63c9dcb581f8fac0980495a783f44241828"}
Nov 03 21:17:03.227094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:03.226709    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9" event={"ID":"533409ce-58a9-4dd3-9767-666c2b90684b","Type":"ContainerStarted","Data":"bece21d8fb50d26ceb618d42673391618446db1cd3c4865f15220ec1118dc1f4"}
Nov 03 21:17:03.342364 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:03.342253    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9" podStartSLOduration=2.342235428 podStartE2EDuration="2.342235428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:17:03.341143863 +0000 UTC m=+2237.792513041" watchObservedRunningTime="2026-11-03 21:17:03.342235428 +0000 UTC m=+2237.793604608"
Nov 03 21:17:04.233826 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:04.233787    5915 generic.go:356] "Generic (PLEG): container finished" podID="533409ce-58a9-4dd3-9767-666c2b90684b" containerID="3f4accf669731367ecd26d1db64ef63c9dcb581f8fac0980495a783f44241828" exitCode=0
Nov 03 21:17:04.234254 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:04.233872    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9" event={"ID":"533409ce-58a9-4dd3-9767-666c2b90684b","Type":"ContainerDied","Data":"3f4accf669731367ecd26d1db64ef63c9dcb581f8fac0980495a783f44241828"}
Nov 03 21:17:05.374777 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.374750    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:05.459264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.459236    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\") pod \"533409ce-58a9-4dd3-9767-666c2b90684b\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") "
Nov 03 21:17:05.459438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.459293    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\") pod \"533409ce-58a9-4dd3-9767-666c2b90684b\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") "
Nov 03 21:17:05.459438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.459383    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\" (UniqueName: \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\") pod \"533409ce-58a9-4dd3-9767-666c2b90684b\" (UID: \"533409ce-58a9-4dd3-9767-666c2b90684b\") "
Nov 03 21:17:05.459667 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.459637    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca" pod "533409ce-58a9-4dd3-9767-666c2b90684b" (UID: "533409ce-58a9-4dd3-9767-666c2b90684b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:17:05.461571 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.461539    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca" pod "533409ce-58a9-4dd3-9767-666c2b90684b" (UID: "533409ce-58a9-4dd3-9767-666c2b90684b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:17:05.461734 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.461714    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk" pod "533409ce-58a9-4dd3-9767-666c2b90684b" (UID: "533409ce-58a9-4dd3-9767-666c2b90684b"). InnerVolumeSpecName "kube-api-access-wj6sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:17:05.560847 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.560784    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:05.560847 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.560806    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/533409ce-58a9-4dd3-9767-666c2b90684b-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:05.560847 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:05.560816    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj6sk\" (UniqueName: \"kubernetes.io/projected/533409ce-58a9-4dd3-9767-666c2b90684b-kube-api-access-wj6sk\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:06.246902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:06.246859    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9" event={"ID":"533409ce-58a9-4dd3-9767-666c2b90684b","Type":"ContainerDied","Data":"bece21d8fb50d26ceb618d42673391618446db1cd3c4865f15220ec1118dc1f4"}
Nov 03 21:17:06.246902 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:06.246903    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"
Nov 03 21:17:06.247120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:06.246919    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bece21d8fb50d26ceb618d42673391618446db1cd3c4865f15220ec1118dc1f4"
Nov 03 21:17:07.002485 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:17:07.002450    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:17:10.845939 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:10.845909    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:26.712931 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:26.712845    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:26.863875 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:26.863843    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:37.003538 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:17:37.003493    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:17:37.192827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.192796    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"]
Nov 03 21:17:37.193214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.193202    5915 cpu_manager.go:414] "containerMap: removing container" podUID="533409ce-58a9-4dd3-9767-666c2b90684b" containerName="revoke-keys"
Nov 03 21:17:37.193263 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.193216    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="533409ce-58a9-4dd3-9767-666c2b90684b" containerName="revoke-keys"
Nov 03 21:17:37.193330 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.193321    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="533409ce-58a9-4dd3-9767-666c2b90684b" containerName="revoke-keys"
Nov 03 21:17:37.196897 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.196876    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.248604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.248542    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:17:37.289059 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.288978    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk9j7\" (UniqueName: \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.289230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.289171    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.289230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.289199    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.298079 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.298057    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:17:37.298208 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.298119    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:17:37.300584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.300552    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"]
Nov 03 21:17:37.347524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.347505    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:17:37.390714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.390686    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk9j7\" (UniqueName: \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.390828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.390785    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.390828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.390808    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.391509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.391488    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.393205 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.393181    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.451666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.451636    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk9j7\" (UniqueName: \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\") pod \"maas-api-revoke-sub-046e1343-5zsgx\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.510495 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.510472    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:37.885910 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:37.885886    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"]
Nov 03 21:17:37.887869 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:17:37.887835    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565751ae_4ae7_4b9b_867a_619e400ea2ab.slice/crio-df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1 WatchSource:0}: Error finding container df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1: Status 404 returned error can't find the container with id df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1
Nov 03 21:17:38.486881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:38.486804    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx" event={"ID":"565751ae-4ae7-4b9b-867a-619e400ea2ab","Type":"ContainerStarted","Data":"5d0b7550074aba51493fdae5b7c104e9e9617dddd7f51724e4dee207c8ec8e75"}
Nov 03 21:17:38.486881 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:38.486854    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx" event={"ID":"565751ae-4ae7-4b9b-867a-619e400ea2ab","Type":"ContainerStarted","Data":"df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1"}
Nov 03 21:17:38.603597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:38.603527    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx" podStartSLOduration=1.60351182 podStartE2EDuration="1.60351182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:17:38.6007032 +0000 UTC m=+2273.052072376" watchObservedRunningTime="2026-11-03 21:17:38.60351182 +0000 UTC m=+2273.054881020"
Nov 03 21:17:39.495118 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:39.495082    5915 generic.go:356] "Generic (PLEG): container finished" podID="565751ae-4ae7-4b9b-867a-619e400ea2ab" containerID="5d0b7550074aba51493fdae5b7c104e9e9617dddd7f51724e4dee207c8ec8e75" exitCode=0
Nov 03 21:17:39.495501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:39.495175    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx" event={"ID":"565751ae-4ae7-4b9b-867a-619e400ea2ab","Type":"ContainerDied","Data":"5d0b7550074aba51493fdae5b7c104e9e9617dddd7f51724e4dee207c8ec8e75"}
Nov 03 21:17:40.644815 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.644790    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:40.724994 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.724966    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\") pod \"565751ae-4ae7-4b9b-867a-619e400ea2ab\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") "
Nov 03 21:17:40.725132 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.725050    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\" (UniqueName: \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\") pod \"565751ae-4ae7-4b9b-867a-619e400ea2ab\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") "
Nov 03 21:17:40.725198 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.725185    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\") pod \"565751ae-4ae7-4b9b-867a-619e400ea2ab\" (UID: \"565751ae-4ae7-4b9b-867a-619e400ea2ab\") "
Nov 03 21:17:40.725312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.725287    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca" pod "565751ae-4ae7-4b9b-867a-619e400ea2ab" (UID: "565751ae-4ae7-4b9b-867a-619e400ea2ab"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:17:40.725461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.725446    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:40.727230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.727205    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca" pod "565751ae-4ae7-4b9b-867a-619e400ea2ab" (UID: "565751ae-4ae7-4b9b-867a-619e400ea2ab"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:17:40.727354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.727336    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7" pod "565751ae-4ae7-4b9b-867a-619e400ea2ab" (UID: "565751ae-4ae7-4b9b-867a-619e400ea2ab"). InnerVolumeSpecName "kube-api-access-hk9j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:17:40.826720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.826639    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/565751ae-4ae7-4b9b-867a-619e400ea2ab-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:40.826720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:40.826670    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk9j7\" (UniqueName: \"kubernetes.io/projected/565751ae-4ae7-4b9b-867a-619e400ea2ab-kube-api-access-hk9j7\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:41.507980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:41.507949    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"
Nov 03 21:17:41.508193 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:41.507990    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx" event={"ID":"565751ae-4ae7-4b9b-867a-619e400ea2ab","Type":"ContainerDied","Data":"df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1"}
Nov 03 21:17:41.508193 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:41.508045    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df0f7ee7c5016c5f39427df35ee2f3548ef9f9eb7ba70b960b95920a903660a1"
Nov 03 21:17:44.252273 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:44.252240    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:47.974891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.974856    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"]
Nov 03 21:17:47.975475 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.975454    5915 cpu_manager.go:414] "containerMap: removing container" podUID="565751ae-4ae7-4b9b-867a-619e400ea2ab" containerName="revoke-keys"
Nov 03 21:17:47.975587 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.975477    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="565751ae-4ae7-4b9b-867a-619e400ea2ab" containerName="revoke-keys"
Nov 03 21:17:47.975651 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.975639    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="565751ae-4ae7-4b9b-867a-619e400ea2ab" containerName="revoke-keys"
Nov 03 21:17:47.978871 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.978855    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:47.999997 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:47.999962    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.000131 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.000106    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc9v9\" (UniqueName: \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.000215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.000198    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.080266 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.080234    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:17:48.080266 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.080240    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:17:48.080266 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.080266    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:17:48.080515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.080307    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:17:48.083110 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.083091    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"]
Nov 03 21:17:48.100742 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.100720    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.100861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.100781    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc9v9\" (UniqueName: \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.100861 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.100838    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.101494 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.101477    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.103148 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.103130    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.161239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.161219    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc9v9\" (UniqueName: \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\") pod \"maas-api-revoke-sub-d3fba056-pgn7v\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.292316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.292246    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:48.665877 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:48.665851    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"]
Nov 03 21:17:48.668186 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:17:48.668159    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c403255_9292_4ab3_85d5_1cde32d01d94.slice/crio-6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b WatchSource:0}: Error finding container 6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b: Status 404 returned error can't find the container with id 6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b
Nov 03 21:17:49.563791 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:49.563758    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v" event={"ID":"0c403255-9292-4ab3-85d5-1cde32d01d94","Type":"ContainerStarted","Data":"f223eb229aaf91c9aeaa5ff5ff67128f32e29c6afc4fac45722fb51f4911fac9"}
Nov 03 21:17:49.564185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:49.563807    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v" event={"ID":"0c403255-9292-4ab3-85d5-1cde32d01d94","Type":"ContainerStarted","Data":"6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b"}
Nov 03 21:17:49.679855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:49.679810    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v" podStartSLOduration=2.6797958680000002 podStartE2EDuration="2.679795868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:17:49.677858915 +0000 UTC m=+2284.129228111" watchObservedRunningTime="2026-11-03 21:17:49.679795868 +0000 UTC m=+2284.131165046"
Nov 03 21:17:50.576381 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:50.576328    5915 generic.go:356] "Generic (PLEG): container finished" podID="0c403255-9292-4ab3-85d5-1cde32d01d94" containerID="f223eb229aaf91c9aeaa5ff5ff67128f32e29c6afc4fac45722fb51f4911fac9" exitCode=0
Nov 03 21:17:50.576916 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:50.576404    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v" event={"ID":"0c403255-9292-4ab3-85d5-1cde32d01d94","Type":"ContainerDied","Data":"f223eb229aaf91c9aeaa5ff5ff67128f32e29c6afc4fac45722fb51f4911fac9"}
Nov 03 21:17:52.540809 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.540780    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:52.593961 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.593926    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\") pod \"0c403255-9292-4ab3-85d5-1cde32d01d94\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") "
Nov 03 21:17:52.594261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.594239    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\" (UniqueName: \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\") pod \"0c403255-9292-4ab3-85d5-1cde32d01d94\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") "
Nov 03 21:17:52.594496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.594479    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\") pod \"0c403255-9292-4ab3-85d5-1cde32d01d94\" (UID: \"0c403255-9292-4ab3-85d5-1cde32d01d94\") "
Nov 03 21:17:52.594994 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.594959    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca" pod "0c403255-9292-4ab3-85d5-1cde32d01d94" (UID: "0c403255-9292-4ab3-85d5-1cde32d01d94"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:17:52.625814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.625680    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca" pod "0c403255-9292-4ab3-85d5-1cde32d01d94" (UID: "0c403255-9292-4ab3-85d5-1cde32d01d94"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:17:52.625814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.625785    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9" pod "0c403255-9292-4ab3-85d5-1cde32d01d94" (UID: "0c403255-9292-4ab3-85d5-1cde32d01d94"). InnerVolumeSpecName "kube-api-access-xc9v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:17:52.796588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.795865    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:52.796588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.795902    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc9v9\" (UniqueName: \"kubernetes.io/projected/0c403255-9292-4ab3-85d5-1cde32d01d94-kube-api-access-xc9v9\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:52.796588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.795922    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0c403255-9292-4ab3-85d5-1cde32d01d94-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:17:52.797691 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.797656    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v" event={"ID":"0c403255-9292-4ab3-85d5-1cde32d01d94","Type":"ContainerDied","Data":"6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b"}
Nov 03 21:17:52.797869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.797853    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6305bea4ef2d21956276b894299edaad45f8ee55850044365df49ed66506740b"
Nov 03 21:17:52.798051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.798036    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"
Nov 03 21:17:52.841596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.838664    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:17:52.841596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:52.838723    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:17:54.471092 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:54.471061    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:17:54.624056 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:17:54.624014    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:18:07.003942 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:18:07.003904    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:18:17.307096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:18:17.307063    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:18:17.468854 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:18:17.468825    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:18:25.006645 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:18:25.006581    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:18:37.003769 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:18:37.003735    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:18:44.602784 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:18:44.602715    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:19:07.003630 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:19:07.003515    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:19:12.134945 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:12.134909    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"]
Nov 03 21:19:12.178437 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:12.178401    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4e35f83-jtbgx"]
Nov 03 21:19:13.695230 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:13.695190    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4912afc9-a302-4518-88e2-6a23c1539fe4" path="/var/lib/kubelet/pods/4912afc9-a302-4518-88e2-6a23c1539fe4/volumes"
Nov 03 21:19:25.355254 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:25.355214    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:19:37.002571 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:19:37.002517    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:19:44.931447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:44.931382    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:19:46.310592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:46.310543    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:19:46.310592 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:46.310592    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:19:46.324857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:46.324832    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:19:46.324996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:19:46.324832    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:20:05.491124 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:05.491092    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:20:07.003256 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:20:07.003221    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:20:22.861705 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:22.861661    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:20:22.862395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:22.862361    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:20:25.691804 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:25.691733    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:20:37.004877 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:20:37.004844    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:20:45.355986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:45.355941    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:20:51.126220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:20:51.126184    5915 scope.go:122] "RemoveContainer" containerID="5b25ec8b64c92471ac83edc49eb22584e299949353c964fa6ce44b960603847c"
Nov 03 21:21:05.777248 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:05.777215    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:21:07.003106 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:21:07.003072    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:21:22.064595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:22.063072    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"]
Nov 03 21:21:22.098583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:22.096937    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c1e271a9-j54gn"]
Nov 03 21:21:22.845699 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:22.845663    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:21:22.845996 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:22.845963    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8" podUID="1db44974-2a96-495d-bccd-290898b0c2af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:21:23.679989 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:23.679958    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e" path="/var/lib/kubelet/pods/7efcff1e-8fdc-4cfe-91c1-f0ece90a9a2e/volumes"
Nov 03 21:21:26.142267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:26.142231    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:21:37.003960 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:21:37.003918    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:21:44.519584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:44.507050    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:21:45.717010 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:45.716826    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:21:54.053189 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:54.053151    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"]
Nov 03 21:21:54.104631 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:54.104587    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-vk94f"]
Nov 03 21:21:55.680313 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:21:55.680240    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4731beef-8a2d-47b2-9235-76e3f6b2c1f4" path="/var/lib/kubelet/pods/4731beef-8a2d-47b2-9235-76e3f6b2c1f4/volumes"
Nov 03 21:22:06.047669 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:06.047637    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"]
Nov 03 21:22:06.097486 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:06.097451    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-4z4z9"]
Nov 03 21:22:06.123351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:06.123321    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:07.003226 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:22:07.003195    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:22:07.680898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:07.680865    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="533409ce-58a9-4dd3-9767-666c2b90684b" path="/var/lib/kubelet/pods/533409ce-58a9-4dd3-9767-666c2b90684b/volumes"
Nov 03 21:22:26.392550 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:26.392512    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:30.867950 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:30.867917    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:31.813302 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:31.813270    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:31.966278 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:31.966248    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:36.087579 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:36.087537    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:36.259703 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:36.259660    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:36.446835 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:36.446757    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:37.002918 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:22:37.002886    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:22:39.404066 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.404025    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:39.580239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.580187    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:39.823433 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.823403    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:22:39.823927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.823902    5915 cpu_manager.go:414] "containerMap: removing container" podUID="0c403255-9292-4ab3-85d5-1cde32d01d94" containerName="revoke-keys"
Nov 03 21:22:39.823927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.823924    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c403255-9292-4ab3-85d5-1cde32d01d94" containerName="revoke-keys"
Nov 03 21:22:39.824143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.824124    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c403255-9292-4ab3-85d5-1cde32d01d94" containerName="revoke-keys"
Nov 03 21:22:39.827818 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.827802    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:39.856349 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.856330    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6fhx\" (UniqueName: \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:39.856488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.856466    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:39.879762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.879736    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-96c0ab\""
Nov 03 21:22:39.961700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.956754    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:22:39.961700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.957834    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:39.961700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.957936    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6fhx\" (UniqueName: \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:39.961700 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:39.961163    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:40.034584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.034531    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6fhx\" (UniqueName: \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\") pod \"payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:40.140773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.140709    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:40.516672 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.516640    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:22:40.518746 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:22:40.518713    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bbda13d_2439_4005_a933_4c94cc4971ae.slice/crio-6b25e614a0e918fdcb4ed398b966e77becd793cdcb4d7919c1c84de3a50d73b8 WatchSource:0}: Error finding container 6b25e614a0e918fdcb4ed398b966e77becd793cdcb4d7919c1c84de3a50d73b8: Status 404 returned error can't find the container with id 6b25e614a0e918fdcb4ed398b966e77becd793cdcb4d7919c1c84de3a50d73b8
Nov 03 21:22:40.990432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.990394    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" event={"ID":"3bbda13d-2439-4005-a933-4c94cc4971ae","Type":"ContainerStarted","Data":"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"}
Nov 03 21:22:40.990631 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.990459    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:40.990631 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:40.990483    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" event={"ID":"3bbda13d-2439-4005-a933-4c94cc4971ae","Type":"ContainerStarted","Data":"6b25e614a0e918fdcb4ed398b966e77becd793cdcb4d7919c1c84de3a50d73b8"}
Nov 03 21:22:41.094026 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:41.093996    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"]
Nov 03 21:22:41.096106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:41.096080    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-5zsgx"]
Nov 03 21:22:41.680811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:41.680769    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="565751ae-4ae7-4b9b-867a-619e400ea2ab" path="/var/lib/kubelet/pods/565751ae-4ae7-4b9b-867a-619e400ea2ab/volumes"
Nov 03 21:22:42.462354 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:42.462318    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:42.618165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:42.618131    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:42.770012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:42.769925    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:46.073687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:46.073656    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:47.421746 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:47.421708    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:51.162055 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:51.162025    5915 scope.go:122] "RemoveContainer" containerID="6f213688c30f261f68ecab680f0fef337e36dd5e2d6e1690cd07ab9e4acdd1d8"
Nov 03 21:22:51.173658 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:51.173623    5915 scope.go:122] "RemoveContainer" containerID="141f4fcf8079d63103c5a5a295b589dc12f24bed79e659cd7ce8a029f900fb0f"
Nov 03 21:22:51.999683 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:51.999655    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:22:52.046891 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:52.046835    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" podStartSLOduration=13.046817467 podStartE2EDuration="13.046817467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:22:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:22:41.141229763 +0000 UTC m=+2575.592598940" watchObservedRunningTime="2026-11-03 21:22:52.046817467 +0000 UTC m=+2586.498186644"
Nov 03 21:22:52.049164 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:52.049133    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"]
Nov 03 21:22:52.098531 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:52.098506    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-pgn7v"]
Nov 03 21:22:52.326264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:52.326179    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:22:53.680609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:53.680582    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c403255-9292-4ab3-85d5-1cde32d01d94" path="/var/lib/kubelet/pods/0c403255-9292-4ab3-85d5-1cde32d01d94/volumes"
Nov 03 21:22:57.111810 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:22:57.111766    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:02.130613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:02.130549    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:04.194613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:04.194579    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:06.484725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:06.484690    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:06.839957 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:06.839870    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:07.003478 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:23:07.003435    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:23:10.843438 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:10.843379    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:11.422265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:11.422232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:11.576686 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:11.576654    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:11.731716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:11.731681    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:14.727349 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:14.727315    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:16.430351 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:16.430318    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:21.233003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:21.232969    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:26.010502 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:26.010409    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:26.755782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:26.755746    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:30.816636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:30.816603    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:35.538524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:35.538490    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:37.002970 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:23:37.002936    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:23:40.445289 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:40.445256    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:45.138112 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:45.138077    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:46.419234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:46.419203    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:50.014291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:50.014261    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:51.205395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:51.205368    5915 scope.go:122] "RemoveContainer" containerID="5d0b7550074aba51493fdae5b7c104e9e9617dddd7f51724e4dee207c8ec8e75"
Nov 03 21:23:51.214674 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:51.214650    5915 scope.go:122] "RemoveContainer" containerID="f223eb229aaf91c9aeaa5ff5ff67128f32e29c6afc4fac45722fb51f4911fac9"
Nov 03 21:23:51.223264 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:51.223236    5915 scope.go:122] "RemoveContainer" containerID="3f4accf669731367ecd26d1db64ef63c9dcb581f8fac0980495a783f44241828"
Nov 03 21:23:54.815655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:54.815619    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:23:59.715573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:23:59.715525    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:04.513468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:04.513437    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:06.814458 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:06.814430    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:07.003931 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:24:07.003898    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:24:09.324103 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:09.324068    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:12.943074 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:12.943041    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:14.286938 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:14.286903    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:17.997857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:17.997823    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:19.501332 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:19.501294    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:24.223160 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:24.223122    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:25.090137 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:25.090098    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:24:25.351337 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:25.351238    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:25.495220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:25.495187    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:27.096998 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:27.096967    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:28.708430 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:28.708396    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:29.940261 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:29.940229    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:30.091165 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:30.091120    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing" containerID="cri-o://b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f" gracePeriod=25
Nov 03 21:24:30.219429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:30.219395    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:30.952864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:30.952834    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:31.997615 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:31.997582    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:24:37.003470 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:24:37.003438    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:24:37.467385 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:37.467330    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:37.634588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:37.634538    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:37.834320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:37.834240    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:41.998682 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:41.998648    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 03 21:24:43.736304 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:43.736270    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:45.196892 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:45.196860    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:46.352235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:46.352202    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:24:46.355428 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:46.355396    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:24:46.366923 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:46.366901    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:24:46.369729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:46.369709    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:24:46.763687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:46.763652    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:50.361239 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.361184    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:24:50.477236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.477204    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\") pod \"3bbda13d-2439-4005-a933-4c94cc4971ae\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") "
Nov 03 21:24:50.477441 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.477362    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\" (UniqueName: \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\") pod \"3bbda13d-2439-4005-a933-4c94cc4971ae\" (UID: \"3bbda13d-2439-4005-a933-4c94cc4971ae\") "
Nov 03 21:24:50.477883 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.477825    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume" pod "3bbda13d-2439-4005-a933-4c94cc4971ae" (UID: "3bbda13d-2439-4005-a933-4c94cc4971ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:24:50.480121 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.480093    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx" pod "3bbda13d-2439-4005-a933-4c94cc4971ae" (UID: "3bbda13d-2439-4005-a933-4c94cc4971ae"). InnerVolumeSpecName "kube-api-access-g6fhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:24:50.578741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.578705    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6fhx\" (UniqueName: \"kubernetes.io/projected/3bbda13d-2439-4005-a933-4c94cc4971ae-kube-api-access-g6fhx\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:50.578741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.578737    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3bbda13d-2439-4005-a933-4c94cc4971ae-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:50.752249 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.752211    5915 generic.go:356] "Generic (PLEG): container finished" podID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerID="b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f" exitCode=0
Nov 03 21:24:50.752399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.752259    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"
Nov 03 21:24:50.752399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.752288    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" event={"ID":"3bbda13d-2439-4005-a933-4c94cc4971ae","Type":"ContainerDied","Data":"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"}
Nov 03 21:24:50.752399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.752345    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f" event={"ID":"3bbda13d-2439-4005-a933-4c94cc4971ae","Type":"ContainerDied","Data":"6b25e614a0e918fdcb4ed398b966e77becd793cdcb4d7919c1c84de3a50d73b8"}
Nov 03 21:24:50.752399 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.752366    5915 scope.go:122] "RemoveContainer" containerID="b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"
Nov 03 21:24:50.762181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.762156    5915 scope.go:122] "RemoveContainer" containerID="b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"
Nov 03 21:24:50.762438 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:24:50.762418    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f\": container with ID starting with b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f not found: ID does not exist" containerID="b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"
Nov 03 21:24:50.762490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.762447    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f"} err="failed to get container status \"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f\": rpc error: code = NotFound desc = could not find container \"b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f\": container with ID starting with b6e77025938789f51ecc7b155db61e7e9722b0be7e1b94e043106aa9001b856f not found: ID does not exist"
Nov 03 21:24:50.774237 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.774218    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:24:50.914882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.914850    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:24:50.919128 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:50.919103    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-96c0ab-6bc5dtx79f"]
Nov 03 21:24:51.680327 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:24:51.680249    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" path="/var/lib/kubelet/pods/3bbda13d-2439-4005-a933-4c94cc4971ae/volumes"
Nov 03 21:25:07.003229 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:25:07.003148    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:25:07.215788 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:25:07.215753    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:25:27.448277 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:25:27.448246    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:25:37.002352 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:25:37.002315    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:25:47.120514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:25:47.120484    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:26:03.288894 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:03.288862    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:26:07.002527 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:26:07.002488    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:26:07.510182 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:07.510149    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:26:27.849293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:27.849219    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:26:37.002648 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:26:37.002613    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:26:41.760773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:41.760733    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:26:41.761405 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:41.761370    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s" podUID="fb0d68f7-486e-44d7-a324-11f4ea590bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:26:43.468518 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.468480    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"]
Nov 03 21:26:43.468941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.468931    5915 cpu_manager.go:414] "containerMap: removing container" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing"
Nov 03 21:26:43.468988 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.468943    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing"
Nov 03 21:26:43.469038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.469026    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bbda13d-2439-4005-a933-4c94cc4971ae" containerName="payload-pre-processing"
Nov 03 21:26:43.472019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.472002    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.573448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.573395    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:26:43.573448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.573414    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:26:43.573448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.573402    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:26:43.573448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.573417    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:26:43.576003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.575981    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"]
Nov 03 21:26:43.581521 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.581484    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hk4c\" (UniqueName: \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.581665 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.581551    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.581714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.581667    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.682885 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.682854    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.683033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.682896    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.683033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.682961    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hk4c\" (UniqueName: \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.683483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.683460    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.685356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.685335    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.743129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.743074    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hk4c\" (UniqueName: \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\") pod \"maas-api-revoke-sub-d576e68b-drp44\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:43.784634 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:43.784606    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:44.160578 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:44.160529    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"]
Nov 03 21:26:44.162602 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:26:44.162538    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c58cae7_5bc2_4833_a16f_2a0ada765d31.slice/crio-2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed WatchSource:0}: Error finding container 2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed: Status 404 returned error can't find the container with id 2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed
Nov 03 21:26:44.525299 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:44.525261    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44" event={"ID":"1c58cae7-5bc2-4833-a16f-2a0ada765d31","Type":"ContainerStarted","Data":"071129e6d39656375aeee09087362e930dbc9a0d5fd5d2977f1e1704c6d832c2"}
Nov 03 21:26:44.525770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:44.525358    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44" event={"ID":"1c58cae7-5bc2-4833-a16f-2a0ada765d31","Type":"ContainerStarted","Data":"2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed"}
Nov 03 21:26:44.640717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:44.640667    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44" podStartSLOduration=1.640652483 podStartE2EDuration="1.640652483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:26:44.639208206 +0000 UTC m=+2819.090577384" watchObservedRunningTime="2026-11-03 21:26:44.640652483 +0000 UTC m=+2819.092021661"
Nov 03 21:26:45.533355 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:45.533319    5915 generic.go:356] "Generic (PLEG): container finished" podID="1c58cae7-5bc2-4833-a16f-2a0ada765d31" containerID="071129e6d39656375aeee09087362e930dbc9a0d5fd5d2977f1e1704c6d832c2" exitCode=0
Nov 03 21:26:45.533750 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:45.533375    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44" event={"ID":"1c58cae7-5bc2-4833-a16f-2a0ada765d31","Type":"ContainerDied","Data":"071129e6d39656375aeee09087362e930dbc9a0d5fd5d2977f1e1704c6d832c2"}
Nov 03 21:26:46.669821 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.669794    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:46.711207 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.711182    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\") pod \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") "
Nov 03 21:26:46.711320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.711212    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\" (UniqueName: \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\") pod \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") "
Nov 03 21:26:46.711320 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.711259    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\") pod \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\" (UID: \"1c58cae7-5bc2-4833-a16f-2a0ada765d31\") "
Nov 03 21:26:46.711613 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.711587    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca" pod "1c58cae7-5bc2-4833-a16f-2a0ada765d31" (UID: "1c58cae7-5bc2-4833-a16f-2a0ada765d31"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:26:46.713395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.713376    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca" pod "1c58cae7-5bc2-4833-a16f-2a0ada765d31" (UID: "1c58cae7-5bc2-4833-a16f-2a0ada765d31"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:26:46.713529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.713514    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c" pod "1c58cae7-5bc2-4833-a16f-2a0ada765d31" (UID: "1c58cae7-5bc2-4833-a16f-2a0ada765d31"). InnerVolumeSpecName "kube-api-access-6hk4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:26:46.812416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.812331    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:26:46.812416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.812357    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hk4c\" (UniqueName: \"kubernetes.io/projected/1c58cae7-5bc2-4833-a16f-2a0ada765d31-kube-api-access-6hk4c\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:26:46.812416 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:46.812367    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c58cae7-5bc2-4833-a16f-2a0ada765d31-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:26:47.462880 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:47.462844    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:26:47.546687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:47.546663    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"
Nov 03 21:26:47.546850 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:47.546657    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44" event={"ID":"1c58cae7-5bc2-4833-a16f-2a0ada765d31","Type":"ContainerDied","Data":"2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed"}
Nov 03 21:26:47.546850 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:26:47.546791    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a0a64f925e2b41c8fa746f2835bdb18c96aed5e22e01c9078ab3d3dd4fde2ed"
Nov 03 21:27:03.738723 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:03.738687    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:27:03.739482 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:03.739431    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:27:07.007227 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:27:07.007187    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:27:07.857627 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:07.857581    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:12.429941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.429907    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"]
Nov 03 21:27:12.430422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.430405    5915 cpu_manager.go:414] "containerMap: removing container" podUID="1c58cae7-5bc2-4833-a16f-2a0ada765d31" containerName="revoke-keys"
Nov 03 21:27:12.430475 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.430424    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c58cae7-5bc2-4833-a16f-2a0ada765d31" containerName="revoke-keys"
Nov 03 21:27:12.430524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.430514    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c58cae7-5bc2-4833-a16f-2a0ada765d31" containerName="revoke-keys"
Nov 03 21:27:12.433986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.433962    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.486245 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.486218    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:27:12.495499 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.495468    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.495657 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.495510    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.495657 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.495635    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppcpc\" (UniqueName: \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.535538 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.535507    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:27:12.535701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.535535    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:27:12.535701 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.535582    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:27:12.537867 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.537847    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"]
Nov 03 21:27:12.596749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.596715    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.596893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.596761    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.596893 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.596808    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppcpc\" (UniqueName: \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.597509 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.597461    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.599237 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.599211    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.657569 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.657530    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppcpc\" (UniqueName: \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\") pod \"maas-api-revoke-sub-d2bc93bc-jpxkh\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:12.747444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:12.747416    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:13.120341 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:13.120316    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"]
Nov 03 21:27:13.122188 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:27:13.122154    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94819cb_42b7_4770_bb1d_9402c23ea5ce.slice/crio-9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81 WatchSource:0}: Error finding container 9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81: Status 404 returned error can't find the container with id 9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81
Nov 03 21:27:13.731263 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:13.731229    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh" event={"ID":"e94819cb-42b7-4770-bb1d-9402c23ea5ce","Type":"ContainerStarted","Data":"d187fcb8637ba6c5cf53403fc644cd29ea9c9570973e2aa4d048f81ddd3a1447"}
Nov 03 21:27:13.731686 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:13.731283    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh" event={"ID":"e94819cb-42b7-4770-bb1d-9402c23ea5ce","Type":"ContainerStarted","Data":"9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81"}
Nov 03 21:27:13.849763 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:13.849701    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh" podStartSLOduration=1.849686232 podStartE2EDuration="1.849686232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:13.84705106 +0000 UTC m=+2848.298420238" watchObservedRunningTime="2026-11-03 21:27:13.849686232 +0000 UTC m=+2848.301055408"
Nov 03 21:27:14.738977 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:14.738945    5915 generic.go:356] "Generic (PLEG): container finished" podID="e94819cb-42b7-4770-bb1d-9402c23ea5ce" containerID="d187fcb8637ba6c5cf53403fc644cd29ea9c9570973e2aa4d048f81ddd3a1447" exitCode=0
Nov 03 21:27:14.739371 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:14.739023    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh" event={"ID":"e94819cb-42b7-4770-bb1d-9402c23ea5ce","Type":"ContainerDied","Data":"d187fcb8637ba6c5cf53403fc644cd29ea9c9570973e2aa4d048f81ddd3a1447"}
Nov 03 21:27:15.886374 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:15.886350    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:16.032546 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.032460    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\") pod \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") "
Nov 03 21:27:16.032546 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.032515    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\") pod \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") "
Nov 03 21:27:16.032816 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.032552    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\" (UniqueName: \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\") pod \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\" (UID: \"e94819cb-42b7-4770-bb1d-9402c23ea5ce\") "
Nov 03 21:27:16.032968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.032942    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca" pod "e94819cb-42b7-4770-bb1d-9402c23ea5ce" (UID: "e94819cb-42b7-4770-bb1d-9402c23ea5ce"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:27:16.034798 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.034772    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca" pod "e94819cb-42b7-4770-bb1d-9402c23ea5ce" (UID: "e94819cb-42b7-4770-bb1d-9402c23ea5ce"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:27:16.034914 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.034799    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc" pod "e94819cb-42b7-4770-bb1d-9402c23ea5ce" (UID: "e94819cb-42b7-4770-bb1d-9402c23ea5ce"). InnerVolumeSpecName "kube-api-access-ppcpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:27:16.134340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.134313    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppcpc\" (UniqueName: \"kubernetes.io/projected/e94819cb-42b7-4770-bb1d-9402c23ea5ce-kube-api-access-ppcpc\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:27:16.134340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.134337    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:27:16.134512 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.134348    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e94819cb-42b7-4770-bb1d-9402c23ea5ce-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:27:16.751335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.751305    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"
Nov 03 21:27:16.751528 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.751330    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh" event={"ID":"e94819cb-42b7-4770-bb1d-9402c23ea5ce","Type":"ContainerDied","Data":"9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81"}
Nov 03 21:27:16.751528 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:16.751377    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f110efccd494bf1393c499312642301bdb349005e5018bf66c95c1563740b81"
Nov 03 21:27:24.797828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:24.797792    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:25.386514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:25.386486    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:25.707569 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:25.707515    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:28.134031 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:28.133996    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:29.420586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:29.420533    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:29.593717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:29.593690    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:29.869625 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:29.869595    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:32.718051 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:32.718005    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:32.898385 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:32.898356    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:33.463721 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.463687    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:27:33.464122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.464109    5915 cpu_manager.go:414] "containerMap: removing container" podUID="e94819cb-42b7-4770-bb1d-9402c23ea5ce" containerName="revoke-keys"
Nov 03 21:27:33.464163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.464124    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e94819cb-42b7-4770-bb1d-9402c23ea5ce" containerName="revoke-keys"
Nov 03 21:27:33.464235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.464227    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="e94819cb-42b7-4770-bb1d-9402c23ea5ce" containerName="revoke-keys"
Nov 03 21:27:33.467662 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.467643    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.508831 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.508798    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp4mm\" (UniqueName: \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.508962 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.508846    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.520098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.520076    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-eb1249\""
Nov 03 21:27:33.520202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.520170    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-f4dcec6e-dockercfg-lxt4r\""
Nov 03 21:27:33.572125 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.572096    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:27:33.610481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.610443    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp4mm\" (UniqueName: \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.610673 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.610524    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.611221 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.611204    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.671588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.671537    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp4mm\" (UniqueName: \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\") pod \"payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:33.780516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:33.780436    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:34.156712 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:34.156687    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:27:34.159048 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:27:34.159014    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1b1314_1f7b_49a3_928b_dece1cda610f.slice/crio-fed80c923d4c0ee00dd0ece5df9657c005af10ce624306fb59c7633d63df7245 WatchSource:0}: Error finding container fed80c923d4c0ee00dd0ece5df9657c005af10ce624306fb59c7633d63df7245: Status 404 returned error can't find the container with id fed80c923d4c0ee00dd0ece5df9657c005af10ce624306fb59c7633d63df7245
Nov 03 21:27:34.875924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:34.875884    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" event={"ID":"6a1b1314-1f7b-49a3-928b-dece1cda610f","Type":"ContainerStarted","Data":"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"}
Nov 03 21:27:34.876457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:34.875943    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" event={"ID":"6a1b1314-1f7b-49a3-928b-dece1cda610f","Type":"ContainerStarted","Data":"fed80c923d4c0ee00dd0ece5df9657c005af10ce624306fb59c7633d63df7245"}
Nov 03 21:27:34.876457 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:34.876137    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:34.991286 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:34.991237    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" podStartSLOduration=1.991223164 podStartE2EDuration="1.991223164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:34.990294601 +0000 UTC m=+2869.441663778" watchObservedRunningTime="2026-11-03 21:27:34.991223164 +0000 UTC m=+2869.442592341"
Nov 03 21:27:35.677890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:35.677857    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:35.829850 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:35.829785    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:35.977688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:35.977656    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:37.002766 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:27:37.002730    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:27:40.626188 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:40.626152    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:45.519223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:45.519191    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:45.900878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:45.900800    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:27:47.803407 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:47.803377    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:50.232909 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:50.232874    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:54.221649 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:54.221617    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:54.739440 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:54.739403    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:54.885465 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:54.885429    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:55.028000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:55.027919    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:55.314842 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:55.314757    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:27:58.062767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:27:58.062734    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:00.117597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:00.117549    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:04.925820 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:04.925786    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:07.003469 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:28:07.003434    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:28:08.199070 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:08.199042    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:09.700040 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:09.699993    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:14.510669 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:14.510633    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:19.309655 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:19.309613    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:24.116163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:24.116112    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:28.489468 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:28.489429    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:28.918731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:28.918645    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:33.715604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:33.715549    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:37.002919 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:28:37.002886    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:28:38.425690 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:38.425651    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:42.603050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.603008    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"]
Nov 03 21:28:42.607069 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.607018    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.620210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.620188    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.620344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.620223    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgvqv\" (UniqueName: \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.620344 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.620251    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.659258 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.659236    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:28:42.708937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.708912    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:28:42.708937 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.708930    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:28:42.709122 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.708912    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:28:42.711356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.711335    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"]
Nov 03 21:28:42.721498 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.721474    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.721637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.721524    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgvqv\" (UniqueName: \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.721637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.721608    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.722343 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.722312    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.724075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.724058    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.782280 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.782254    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgvqv\" (UniqueName: \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\") pod \"maas-api-revoke-sub-78a4f552-sl7bs\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:42.920840 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:42.920774    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:43.221806 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:43.221773    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:43.297905 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:43.297876    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"]
Nov 03 21:28:43.303648 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:28:43.303619    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0769a3_5d8d_4591_907f_d093396ebbe0.slice/crio-be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905 WatchSource:0}: Error finding container be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905: Status 404 returned error can't find the container with id be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905
Nov 03 21:28:44.304065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:44.304027    5915 generic.go:356] "Generic (PLEG): container finished" podID="9a0769a3-5d8d-4591-907f-d093396ebbe0" containerID="038896ae5df50dafe3f7dc3dcc5cb0530c7a525bf96b51ed06c7c8ac91c16dce" exitCode=0
Nov 03 21:28:44.304490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:44.304118    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs" event={"ID":"9a0769a3-5d8d-4591-907f-d093396ebbe0","Type":"ContainerDied","Data":"038896ae5df50dafe3f7dc3dcc5cb0530c7a525bf96b51ed06c7c8ac91c16dce"}
Nov 03 21:28:44.304490 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:44.304175    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs" event={"ID":"9a0769a3-5d8d-4591-907f-d093396ebbe0","Type":"ContainerStarted","Data":"be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905"}
Nov 03 21:28:45.455496 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.455469    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:45.547739 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.547704    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\" (UniqueName: \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\") pod \"9a0769a3-5d8d-4591-907f-d093396ebbe0\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") "
Nov 03 21:28:45.547739 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.547745    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\") pod \"9a0769a3-5d8d-4591-907f-d093396ebbe0\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") "
Nov 03 21:28:45.548003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.547836    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\") pod \"9a0769a3-5d8d-4591-907f-d093396ebbe0\" (UID: \"9a0769a3-5d8d-4591-907f-d093396ebbe0\") "
Nov 03 21:28:45.548263 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.548228    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca" pod "9a0769a3-5d8d-4591-907f-d093396ebbe0" (UID: "9a0769a3-5d8d-4591-907f-d093396ebbe0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:28:45.550107 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.550076    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca" pod "9a0769a3-5d8d-4591-907f-d093396ebbe0" (UID: "9a0769a3-5d8d-4591-907f-d093396ebbe0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:28:45.550252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.550160    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv" pod "9a0769a3-5d8d-4591-907f-d093396ebbe0" (UID: "9a0769a3-5d8d-4591-907f-d093396ebbe0"). InnerVolumeSpecName "kube-api-access-pgvqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:28:45.649535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.649450    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgvqv\" (UniqueName: \"kubernetes.io/projected/9a0769a3-5d8d-4591-907f-d093396ebbe0-kube-api-access-pgvqv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:45.649535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.649482    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:45.649535 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:45.649492    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9a0769a3-5d8d-4591-907f-d093396ebbe0-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:46.317469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:46.317438    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"
Nov 03 21:28:46.317469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:46.317450    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs" event={"ID":"9a0769a3-5d8d-4591-907f-d093396ebbe0","Type":"ContainerDied","Data":"be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905"}
Nov 03 21:28:46.317715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:46.317502    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be3ebcd794afb633e29abee836671b0e2dafa3d7f397d31090860d814c38f905"
Nov 03 21:28:48.015927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:48.015895    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:52.813395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:52.813348    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:53.656315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:53.656278    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:54.132173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.132138    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"]
Nov 03 21:28:54.132626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.132611    5915 cpu_manager.go:414] "containerMap: removing container" podUID="9a0769a3-5d8d-4591-907f-d093396ebbe0" containerName="revoke-keys"
Nov 03 21:28:54.132684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.132628    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a0769a3-5d8d-4591-907f-d093396ebbe0" containerName="revoke-keys"
Nov 03 21:28:54.132740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.132730    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a0769a3-5d8d-4591-907f-d093396ebbe0" containerName="revoke-keys"
Nov 03 21:28:54.135959 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.135942    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.224733 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.224695    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.224887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.224742    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f28mk\" (UniqueName: \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.224887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.224826    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.237232 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.237209    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:28:54.237335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.237248    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:28:54.237335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.237297    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:28:54.237411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.237336    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:28:54.239862 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.239839    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"]
Nov 03 21:28:54.326033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.326007    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.326171 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.326102    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.326171 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.326134    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f28mk\" (UniqueName: \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.326724 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.326702    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.328609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.328589    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.386500 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.386435    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f28mk\" (UniqueName: \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\") pod \"maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.448459 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.448432    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:54.821523 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:54.821501    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"]
Nov 03 21:28:54.823750 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:28:54.823719    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e4fe55d_02fe_4dd4_afd6_1ade7916543e.slice/crio-250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a WatchSource:0}: Error finding container 250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a: Status 404 returned error can't find the container with id 250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a
Nov 03 21:28:55.372451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:55.372376    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r" event={"ID":"8e4fe55d-02fe-4dd4-afd6-1ade7916543e","Type":"ContainerStarted","Data":"318c4fc7218dc6bacfee69abca500d1b14df792d148bf0746b16daedad39b928"}
Nov 03 21:28:55.372451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:55.372427    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r" event={"ID":"8e4fe55d-02fe-4dd4-afd6-1ade7916543e","Type":"ContainerStarted","Data":"250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a"}
Nov 03 21:28:56.379340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:56.379302    5915 generic.go:356] "Generic (PLEG): container finished" podID="8e4fe55d-02fe-4dd4-afd6-1ade7916543e" containerID="318c4fc7218dc6bacfee69abca500d1b14df792d148bf0746b16daedad39b928" exitCode=0
Nov 03 21:28:56.379827 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:56.379386    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r" event={"ID":"8e4fe55d-02fe-4dd4-afd6-1ade7916543e","Type":"ContainerDied","Data":"318c4fc7218dc6bacfee69abca500d1b14df792d148bf0746b16daedad39b928"}
Nov 03 21:28:57.526865 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.526843    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:57.658365 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.658301    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\") pod \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") "
Nov 03 21:28:57.658365 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.658335    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\") pod \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") "
Nov 03 21:28:57.658618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.658368    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\" (UniqueName: \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\") pod \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\" (UID: \"8e4fe55d-02fe-4dd4-afd6-1ade7916543e\") "
Nov 03 21:28:57.658753 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.658730    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca" pod "8e4fe55d-02fe-4dd4-afd6-1ade7916543e" (UID: "8e4fe55d-02fe-4dd4-afd6-1ade7916543e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:28:57.660648 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.660614    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca" pod "8e4fe55d-02fe-4dd4-afd6-1ade7916543e" (UID: "8e4fe55d-02fe-4dd4-afd6-1ade7916543e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:28:57.660751 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.660717    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk" pod "8e4fe55d-02fe-4dd4-afd6-1ade7916543e" (UID: "8e4fe55d-02fe-4dd4-afd6-1ade7916543e"). InnerVolumeSpecName "kube-api-access-f28mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:28:57.759451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.759420    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:57.759451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.759444    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:57.759451 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:57.759453    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f28mk\" (UniqueName: \"kubernetes.io/projected/8e4fe55d-02fe-4dd4-afd6-1ade7916543e-kube-api-access-f28mk\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:28:58.010373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.010338    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:58.392432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.392360    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"
Nov 03 21:28:58.392432 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.392383    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r" event={"ID":"8e4fe55d-02fe-4dd4-afd6-1ade7916543e","Type":"ContainerDied","Data":"250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a"}
Nov 03 21:28:58.392641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.392448    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="250aa30e188105a0ba768a4acbafcb82ef7714c62a9c3ae324335bde8ac5950a"
Nov 03 21:28:58.592338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.592298    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:28:58.793340 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:28:58.793302    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:02.811226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:02.811189    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:05.766782 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:05.766749    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:29:06.024312 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:06.024232    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:06.168038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:06.168007    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:07.004509 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:29:07.004477    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:29:07.571335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:07.571300    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:08.542485 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:08.542457    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:09.329537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:09.329503    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:09.473270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:09.473234    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:11.439922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.439859    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" podUID="6a1b1314-1f7b-49a3-928b-dece1cda610f" containerName="payload-pre-processing" containerID="cri-o://a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91" gracePeriod=25
Nov 03 21:29:11.708581 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.708534    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:29:11.754787 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.754757    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:11.813571 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.813540    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\" (UniqueName: \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\") pod \"6a1b1314-1f7b-49a3-928b-dece1cda610f\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") "
Nov 03 21:29:11.813731 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.813640    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\") pod \"6a1b1314-1f7b-49a3-928b-dece1cda610f\" (UID: \"6a1b1314-1f7b-49a3-928b-dece1cda610f\") "
Nov 03 21:29:11.814012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.813990    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume" pod "6a1b1314-1f7b-49a3-928b-dece1cda610f" (UID: "6a1b1314-1f7b-49a3-928b-dece1cda610f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:29:11.815852 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.815828    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm" pod "6a1b1314-1f7b-49a3-928b-dece1cda610f" (UID: "6a1b1314-1f7b-49a3-928b-dece1cda610f"). InnerVolumeSpecName "kube-api-access-lp4mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:11.915396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.915360    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp4mm\" (UniqueName: \"kubernetes.io/projected/6a1b1314-1f7b-49a3-928b-dece1cda610f-kube-api-access-lp4mm\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:11.915396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:11.915389    5915 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a1b1314-1f7b-49a3-928b-dece1cda610f-plugins-config-volume\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:12.475469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.475433    5915 generic.go:356] "Generic (PLEG): container finished" podID="6a1b1314-1f7b-49a3-928b-dece1cda610f" containerID="a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91" exitCode=0
Nov 03 21:29:12.475963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.475476    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" event={"ID":"6a1b1314-1f7b-49a3-928b-dece1cda610f","Type":"ContainerDied","Data":"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"}
Nov 03 21:29:12.475963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.475484    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"
Nov 03 21:29:12.475963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.475529    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z" event={"ID":"6a1b1314-1f7b-49a3-928b-dece1cda610f","Type":"ContainerDied","Data":"fed80c923d4c0ee00dd0ece5df9657c005af10ce624306fb59c7633d63df7245"}
Nov 03 21:29:12.475963 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.475594    5915 scope.go:122] "RemoveContainer" containerID="a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"
Nov 03 21:29:12.485431 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.485413    5915 scope.go:122] "RemoveContainer" containerID="a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"
Nov 03 21:29:12.485729 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:29:12.485711    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91\": container with ID starting with a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91 not found: ID does not exist" containerID="a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"
Nov 03 21:29:12.485813 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.485741    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91"} err="failed to get container status \"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91\": rpc error: code = NotFound desc = could not find container \"a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91\": container with ID starting with a70ed52f2eddeec964fb5fab10c6b94395d36f823804b5bc2b73c31d1bf80a91 not found: ID does not exist"
Nov 03 21:29:12.639299 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.639267    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:29:12.688636 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:12.688605    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-eb1249-744882m85z"]
Nov 03 21:29:13.252437 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:13.252395    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:13.536034 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:13.535961    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:13.682298 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:13.682267    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a1b1314-1f7b-49a3-928b-dece1cda610f" path="/var/lib/kubelet/pods/6a1b1314-1f7b-49a3-928b-dece1cda610f/volumes"
Nov 03 21:29:13.866886 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:13.866812    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"]
Nov 03 21:29:13.916095 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:13.916063    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-eb1249-7nv7r"]
Nov 03 21:29:14.309411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:14.309378    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:15.680887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:15.680850    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e4fe55d-02fe-4dd4-afd6-1ade7916543e" path="/var/lib/kubelet/pods/8e4fe55d-02fe-4dd4-afd6-1ade7916543e/volumes"
Nov 03 21:29:20.386544 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:20.386511    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:20.530960 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:20.530930    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:20.876376 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:20.876341    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:28.562364 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:28.562283    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:28.851163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:28.851069    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:35.987872 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:35.987812    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:36.139444 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:36.139412    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:37.008929 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:29:37.008893    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:29:46.395572 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:46.395526    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:29:46.401138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:46.401112    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:29:46.409404 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:46.409385    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:29:46.418904 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:46.418874    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:29:53.158353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:53.158317    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:29:59.618454 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:29:59.618420    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:00.167593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.167542    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:00.167978 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.167964    5915 cpu_manager.go:414] "containerMap: removing container" podUID="6a1b1314-1f7b-49a3-928b-dece1cda610f" containerName="payload-pre-processing"
Nov 03 21:30:00.168037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.167978    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a1b1314-1f7b-49a3-928b-dece1cda610f" containerName="payload-pre-processing"
Nov 03 21:30:00.168037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.168005    5915 cpu_manager.go:414] "containerMap: removing container" podUID="8e4fe55d-02fe-4dd4-afd6-1ade7916543e" containerName="revoke-keys"
Nov 03 21:30:00.168037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.168011    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e4fe55d-02fe-4dd4-afd6-1ade7916543e" containerName="revoke-keys"
Nov 03 21:30:00.168154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.168076    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a1b1314-1f7b-49a3-928b-dece1cda610f" containerName="payload-pre-processing"
Nov 03 21:30:00.168154 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.168089    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e4fe55d-02fe-4dd4-afd6-1ade7916543e" containerName="revoke-keys"
Nov 03 21:30:00.172439 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.172422    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.223357 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.223335    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.223474 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.223459    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db7qv\" (UniqueName: \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.275773 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.275731    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:30:00.275901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.275773    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:30:00.275901 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.275840    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:30:00.278231 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.278210    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:00.325030 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.325004    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db7qv\" (UniqueName: \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.325181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.325061    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.325714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.325697    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.385543 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.385516    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db7qv\" (UniqueName: \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\") pod \"maas-api-key-cleanup-29895690-rzmnp\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.485469 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.485444    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:00.858384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:00.858355    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:00.859943 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:30:00.859909    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3311ded_04b2_411b_83b6_95ad827af5df.slice/crio-7a1dd2e31bb54efa6206bb77003315f1672f2b2e7a16c55b1e711ba1d6a5af51 WatchSource:0}: Error finding container 7a1dd2e31bb54efa6206bb77003315f1672f2b2e7a16c55b1e711ba1d6a5af51: Status 404 returned error can't find the container with id 7a1dd2e31bb54efa6206bb77003315f1672f2b2e7a16c55b1e711ba1d6a5af51
Nov 03 21:30:01.793588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:01.793477    5915 generic.go:356] "Generic (PLEG): container finished" podID="b3311ded-04b2-411b-83b6-95ad827af5df" containerID="79d619243c77046aef29d483f4f290debe77d41dca6a2b3846c76bdd5d976bb9" exitCode=60
Nov 03 21:30:01.793588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:01.793519    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" event={"ID":"b3311ded-04b2-411b-83b6-95ad827af5df","Type":"ContainerDied","Data":"79d619243c77046aef29d483f4f290debe77d41dca6a2b3846c76bdd5d976bb9"}
Nov 03 21:30:01.793588 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:01.793578    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" event={"ID":"b3311ded-04b2-411b-83b6-95ad827af5df","Type":"ContainerStarted","Data":"7a1dd2e31bb54efa6206bb77003315f1672f2b2e7a16c55b1e711ba1d6a5af51"}
Nov 03 21:30:01.793829 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:01.793815    5915 scope.go:122] "RemoveContainer" containerID="79d619243c77046aef29d483f4f290debe77d41dca6a2b3846c76bdd5d976bb9"
Nov 03 21:30:02.661185 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:02.661154    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:02.800971 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:02.800941    5915 generic.go:356] "Generic (PLEG): container finished" podID="b3311ded-04b2-411b-83b6-95ad827af5df" containerID="fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37" exitCode=60
Nov 03 21:30:02.801106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:02.801004    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" event={"ID":"b3311ded-04b2-411b-83b6-95ad827af5df","Type":"ContainerDied","Data":"fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37"}
Nov 03 21:30:02.801106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:02.801069    5915 scope.go:122] "RemoveContainer" containerID="79d619243c77046aef29d483f4f290debe77d41dca6a2b3846c76bdd5d976bb9"
Nov 03 21:30:02.801270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:02.801257    5915 scope.go:122] "RemoveContainer" containerID="fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37"
Nov 03 21:30:02.801503 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:30:02.801487    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895690-rzmnp_odh-ai-gateway-infra(b3311ded-04b2-411b-83b6-95ad827af5df)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" podUID="b3311ded-04b2-411b-83b6-95ad827af5df"
Nov 03 21:30:03.808514 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:03.808486    5915 scope.go:122] "RemoveContainer" containerID="fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37"
Nov 03 21:30:03.808940 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:30:03.808758    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29895690-rzmnp_odh-ai-gateway-infra(b3311ded-04b2-411b-83b6-95ad827af5df)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" podUID="b3311ded-04b2-411b-83b6-95ad827af5df"
Nov 03 21:30:07.003895 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:30:07.003856    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:30:13.445515 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:13.445478    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:15.680866 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:15.680834    5915 scope.go:122] "RemoveContainer" containerID="fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37"
Nov 03 21:30:16.894930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:16.894853    5915 generic.go:356] "Generic (PLEG): container finished" podID="b3311ded-04b2-411b-83b6-95ad827af5df" containerID="bae543b5fdf5fce2a6ae31561529fb872939043195372168644eff2efa50683f" exitCode=60
Nov 03 21:30:16.894930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:16.894918    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" event={"ID":"b3311ded-04b2-411b-83b6-95ad827af5df","Type":"ContainerDied","Data":"bae543b5fdf5fce2a6ae31561529fb872939043195372168644eff2efa50683f"}
Nov 03 21:30:16.895467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:16.894979    5915 scope.go:122] "RemoveContainer" containerID="fb81ca7654c915400a8b055dcbdcfd2b9fc9c751d25f929b7f6361b3d7d82f37"
Nov 03 21:30:16.895467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:16.895321    5915 scope.go:122] "RemoveContainer" containerID="bae543b5fdf5fce2a6ae31561529fb872939043195372168644eff2efa50683f"
Nov 03 21:30:16.895636 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:30:16.895613    5915 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29895690-rzmnp_odh-ai-gateway-infra(b3311ded-04b2-411b-83b6-95ad827af5df)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" podUID="b3311ded-04b2-411b-83b6-95ad827af5df"
Nov 03 21:30:18.020659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.020625    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:18.166927 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.166901    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:18.304014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.303939    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\" (UniqueName: \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\") pod \"b3311ded-04b2-411b-83b6-95ad827af5df\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") "
Nov 03 21:30:18.304014 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.303988    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\") pod \"b3311ded-04b2-411b-83b6-95ad827af5df\" (UID: \"b3311ded-04b2-411b-83b6-95ad827af5df\") "
Nov 03 21:30:18.304511 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.304466    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca" pod "b3311ded-04b2-411b-83b6-95ad827af5df" (UID: "b3311ded-04b2-411b-83b6-95ad827af5df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:30:18.306764 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.306737    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv" pod "b3311ded-04b2-411b-83b6-95ad827af5df" (UID: "b3311ded-04b2-411b-83b6-95ad827af5df"). InnerVolumeSpecName "kube-api-access-db7qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:30:18.316740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.316694    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:18.405323 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.405281    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db7qv\" (UniqueName: \"kubernetes.io/projected/b3311ded-04b2-411b-83b6-95ad827af5df-kube-api-access-db7qv\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:30:18.405323 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.405310    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b3311ded-04b2-411b-83b6-95ad827af5df-maas-api-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:30:18.528995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.528956    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:18.923396 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.923343    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp" event={"ID":"b3311ded-04b2-411b-83b6-95ad827af5df","Type":"ContainerDied","Data":"7a1dd2e31bb54efa6206bb77003315f1672f2b2e7a16c55b1e711ba1d6a5af51"}
Nov 03 21:30:18.923643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.923425    5915 scope.go:122] "RemoveContainer" containerID="bae543b5fdf5fce2a6ae31561529fb872939043195372168644eff2efa50683f"
Nov 03 21:30:18.923643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:18.923542    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"
Nov 03 21:30:19.091510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:19.091476    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:19.141780 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:19.141743    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29895690-rzmnp"]
Nov 03 21:30:19.685064 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:19.685021    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" path="/var/lib/kubelet/pods/b3311ded-04b2-411b-83b6-95ad827af5df/volumes"
Nov 03 21:30:29.178236 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:29.178198    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:37.004799 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:30:37.004761    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:30:43.962716 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.962678    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"]
Nov 03 21:30:43.963096 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963079    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.963159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963100    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.963159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963145    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.963159 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963150    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.963252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963209    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.963252 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.963218    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:30:43.967635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.967619    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:43.982461 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.982437    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:43.982597 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.982471    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69tmp\" (UniqueName: \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:43.982816 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:43.982791    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.025000 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.024973    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"]
Nov 03 21:30:44.071338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.071309    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:30:44.071338 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.071327    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:30:44.071505 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.071310    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:30:44.071505 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.071310    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:30:44.083930 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.083902    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.084050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.083955    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69tmp\" (UniqueName: \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.084050 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.084016    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.084757 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.084695    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.086570 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.086533    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.144420 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.144398    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69tmp\" (UniqueName: \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\") pod \"maas-api-revoke-sub-452b6d7b-bp648\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.282814 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.282746    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:44.671709 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:44.671680    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"]
Nov 03 21:30:44.672195 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:30:44.672167    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c50605_8214_49e5_b7d6_b7a3785fab38.slice/crio-e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4 WatchSource:0}: Error finding container e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4: Status 404 returned error can't find the container with id e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4
Nov 03 21:30:45.135107 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:45.135074    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648" event={"ID":"35c50605-8214-49e5-b7d6-b7a3785fab38","Type":"ContainerStarted","Data":"cc74f628f3b1399d1465e5310931642de26c87da44331f03d1090410953c9399"}
Nov 03 21:30:45.135467 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:45.135135    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648" event={"ID":"35c50605-8214-49e5-b7d6-b7a3785fab38","Type":"ContainerStarted","Data":"e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4"}
Nov 03 21:30:45.250310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:45.250259    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648" podStartSLOduration=2.250244088 podStartE2EDuration="2.250244088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:45.249146846 +0000 UTC m=+3059.700516024" watchObservedRunningTime="2026-11-03 21:30:45.250244088 +0000 UTC m=+3059.701613265"
Nov 03 21:30:46.149837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:46.149800    5915 generic.go:356] "Generic (PLEG): container finished" podID="35c50605-8214-49e5-b7d6-b7a3785fab38" containerID="cc74f628f3b1399d1465e5310931642de26c87da44331f03d1090410953c9399" exitCode=0
Nov 03 21:30:46.150272 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:46.149882    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648" event={"ID":"35c50605-8214-49e5-b7d6-b7a3785fab38","Type":"ContainerDied","Data":"cc74f628f3b1399d1465e5310931642de26c87da44331f03d1090410953c9399"}
Nov 03 21:30:47.900583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:47.894447    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:48.184678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.181210    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\") pod \"35c50605-8214-49e5-b7d6-b7a3785fab38\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") "
Nov 03 21:30:48.184678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.181302    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\") pod \"35c50605-8214-49e5-b7d6-b7a3785fab38\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") "
Nov 03 21:30:48.184678 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.181343    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\" (UniqueName: \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\") pod \"35c50605-8214-49e5-b7d6-b7a3785fab38\" (UID: \"35c50605-8214-49e5-b7d6-b7a3785fab38\") "
Nov 03 21:30:48.219377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.219052    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca" pod "35c50605-8214-49e5-b7d6-b7a3785fab38" (UID: "35c50605-8214-49e5-b7d6-b7a3785fab38"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:30:48.259324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.259279    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648" event={"ID":"35c50605-8214-49e5-b7d6-b7a3785fab38","Type":"ContainerDied","Data":"e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4"}
Nov 03 21:30:48.260025 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.259983    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6f1845d2d5a63d69b672760cc8a7a81f9037ff3ffc7696ff9df143df48b5aa4"
Nov 03 21:30:48.260180 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.259887    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"
Nov 03 21:30:48.270256 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.270210    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca" pod "35c50605-8214-49e5-b7d6-b7a3785fab38" (UID: "35c50605-8214-49e5-b7d6-b7a3785fab38"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:30:48.288659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.282501    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:30:48.288659 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.282539    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/35c50605-8214-49e5-b7d6-b7a3785fab38-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:30:48.312778 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.312741    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp" pod "35c50605-8214-49e5-b7d6-b7a3785fab38" (UID: "35c50605-8214-49e5-b7d6-b7a3785fab38"). InnerVolumeSpecName "kube-api-access-69tmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:30:48.384583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.384191    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69tmp\" (UniqueName: \"kubernetes.io/projected/35c50605-8214-49e5-b7d6-b7a3785fab38-kube-api-access-69tmp\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:30:48.751582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.750669    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:30:48.751582 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:48.750734    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z" podUID="d48e0d98-751a-41ff-b3ca-a80292e6c9f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:30:53.506609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:53.506535    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:30:56.748138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:30:56.748053    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:07.002920 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:31:07.002876    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:31:13.082610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:13.082540    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:13.908506 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:13.908466    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:17.184200 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184171    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"]
Nov 03 21:31:17.184610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184595    5915 cpu_manager.go:414] "containerMap: removing container" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:31:17.184610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184608    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:31:17.184689 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184631    5915 cpu_manager.go:414] "containerMap: removing container" podUID="35c50605-8214-49e5-b7d6-b7a3785fab38" containerName="revoke-keys"
Nov 03 21:31:17.184689 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184636    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35c50605-8214-49e5-b7d6-b7a3785fab38" containerName="revoke-keys"
Nov 03 21:31:17.184761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184713    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="35c50605-8214-49e5-b7d6-b7a3785fab38" containerName="revoke-keys"
Nov 03 21:31:17.184761 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.184727    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3311ded-04b2-411b-83b6-95ad827af5df" containerName="cleanup"
Nov 03 21:31:17.187911 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.187894    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.240323 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.240296    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:31:17.289218 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.289191    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:31:17.289395 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.289376    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:31:17.289456 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.289442    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:31:17.291946 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.291926    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"]
Nov 03 21:31:17.317187 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.317163    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.317308 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.317193    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.317308 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.317216    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmkbs\" (UniqueName: \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.418549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.418520    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.418717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.418584    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.418717 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.418621    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmkbs\" (UniqueName: \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.419310 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.419261    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.421053 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.421028    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.479549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.479518    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmkbs\" (UniqueName: \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\") pod \"maas-api-revoke-sub-3e31a506-btv86\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.500767 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.500744    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:17.877740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:17.877692    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"]
Nov 03 21:31:17.880904 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:31:17.880860    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c37113_33be_4faf_b6e6_6e0ec4c01e56.slice/crio-f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583 WatchSource:0}: Error finding container f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583: Status 404 returned error can't find the container with id f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583
Nov 03 21:31:18.491874 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:18.491813    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86" event={"ID":"a8c37113-33be-4faf-b6e6-6e0ec4c01e56","Type":"ContainerStarted","Data":"536d6c7f62948299059d7cb24d971f138bfab7d4204bbde8929c7dbfad6b8e34"}
Nov 03 21:31:18.492384 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:18.491932    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86" event={"ID":"a8c37113-33be-4faf-b6e6-6e0ec4c01e56","Type":"ContainerStarted","Data":"f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583"}
Nov 03 21:31:18.608762 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:18.608632    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86" podStartSLOduration=1.6086121549999999 podStartE2EDuration="1.608612155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:31:18.60640244 +0000 UTC m=+3093.057771617" watchObservedRunningTime="2026-11-03 21:31:18.608612155 +0000 UTC m=+3093.059981333"
Nov 03 21:31:19.514583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:19.513895    5915 generic.go:356] "Generic (PLEG): container finished" podID="a8c37113-33be-4faf-b6e6-6e0ec4c01e56" containerID="536d6c7f62948299059d7cb24d971f138bfab7d4204bbde8929c7dbfad6b8e34" exitCode=0
Nov 03 21:31:19.514583 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:19.514109    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86" event={"ID":"a8c37113-33be-4faf-b6e6-6e0ec4c01e56","Type":"ContainerDied","Data":"536d6c7f62948299059d7cb24d971f138bfab7d4204bbde8929c7dbfad6b8e34"}
Nov 03 21:31:20.120584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.119658    5915 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:31:20.120584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.119736    5915 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-xxsms" podUID="4d76e840-1ed9-42ef-b9d1-2c889b992835" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:31:20.710447 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.710420    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:20.785465 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.785397    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\") pod \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") "
Nov 03 21:31:20.785635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.785465    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\" (UniqueName: \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\") pod \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") "
Nov 03 21:31:20.785635 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.785606    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\") pod \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\" (UID: \"a8c37113-33be-4faf-b6e6-6e0ec4c01e56\") "
Nov 03 21:31:20.785803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.785776    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca" pod "a8c37113-33be-4faf-b6e6-6e0ec4c01e56" (UID: "a8c37113-33be-4faf-b6e6-6e0ec4c01e56"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:31:20.785972 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.785956    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:20.787663 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.787643    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs" pod "a8c37113-33be-4faf-b6e6-6e0ec4c01e56" (UID: "a8c37113-33be-4faf-b6e6-6e0ec4c01e56"). InnerVolumeSpecName "kube-api-access-zmkbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:31:20.787770 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.787696    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca" pod "a8c37113-33be-4faf-b6e6-6e0ec4c01e56" (UID: "a8c37113-33be-4faf-b6e6-6e0ec4c01e56"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:31:20.887270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.887235    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmkbs\" (UniqueName: \"kubernetes.io/projected/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-kube-api-access-zmkbs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:20.887270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:20.887263    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a8c37113-33be-4faf-b6e6-6e0ec4c01e56-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:21.538830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:21.538801    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"
Nov 03 21:31:21.538830 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:21.538813    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86" event={"ID":"a8c37113-33be-4faf-b6e6-6e0ec4c01e56","Type":"ContainerDied","Data":"f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583"}
Nov 03 21:31:21.539039 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:21.538857    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4b3850f3997263b825dc8241831691c4e0c09eb38e6842f1c6eb84b0ea92583"
Nov 03 21:31:29.551197 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:29.551159    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:37.002606 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:31:37.002543    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:31:47.055887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:47.055842    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"]
Nov 03 21:31:47.156584 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:47.148979    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-drp44"]
Nov 03 21:31:47.680603 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:47.680551    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c58cae7-5bc2-4833-a16f-2a0ada765d31" path="/var/lib/kubelet/pods/1c58cae7-5bc2-4833-a16f-2a0ada765d31/volumes"
Nov 03 21:31:54.529380 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.529342    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"]
Nov 03 21:31:54.529993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.529972    5915 cpu_manager.go:414] "containerMap: removing container" podUID="a8c37113-33be-4faf-b6e6-6e0ec4c01e56" containerName="revoke-keys"
Nov 03 21:31:54.530091 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.529996    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8c37113-33be-4faf-b6e6-6e0ec4c01e56" containerName="revoke-keys"
Nov 03 21:31:54.530170 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.530154    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8c37113-33be-4faf-b6e6-6e0ec4c01e56" containerName="revoke-keys"
Nov 03 21:31:54.533642 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.533622    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.587898 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.587873    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-77vtd\""
Nov 03 21:31:54.601618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.601592    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.601752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.601654    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.601839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.601772    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5xs\" (UniqueName: \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.637790 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.637759    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:31:54.638027 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.638009    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:31:54.640267 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.640248    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"]
Nov 03 21:31:54.660077 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.660054    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 03 21:31:54.703033 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.703004    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.703223 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.703193    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.703313 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.703289    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5xs\" (UniqueName: \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.703968 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.703947    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.705712 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.705695    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.763702 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.763676    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5xs\" (UniqueName: \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\") pod \"maas-api-revoke-sub-d4735150-zv2kj\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:54.846840 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:54.846782    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:55.218769 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:55.218739    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"]
Nov 03 21:31:55.220393 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:31:55.220358    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3eb0004_2902_4a9a_8bc3_2b54fc65ad55.slice/crio-ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb WatchSource:0}: Error finding container ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb: Status 404 returned error can't find the container with id ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb
Nov 03 21:31:55.837656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:55.837571    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj" event={"ID":"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55","Type":"ContainerStarted","Data":"1cae02fb1e0954e02874d71d20b023fb8a9e728de257bb19edf89a7a86385b82"}
Nov 03 21:31:55.837656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:55.837624    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj" event={"ID":"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55","Type":"ContainerStarted","Data":"ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb"}
Nov 03 21:31:56.001632 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:56.001589    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj" podStartSLOduration=2.001544868 podStartE2EDuration="2.001544868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:31:55.951764063 +0000 UTC m=+3130.403133241" watchObservedRunningTime="2026-11-03 21:31:56.001544868 +0000 UTC m=+3130.452914044"
Nov 03 21:31:56.845422 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:56.845392    5915 generic.go:356] "Generic (PLEG): container finished" podID="d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" containerID="1cae02fb1e0954e02874d71d20b023fb8a9e728de257bb19edf89a7a86385b82" exitCode=0
Nov 03 21:31:56.845835 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:56.845443    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj" event={"ID":"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55","Type":"ContainerDied","Data":"1cae02fb1e0954e02874d71d20b023fb8a9e728de257bb19edf89a7a86385b82"}
Nov 03 21:31:58.010586 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.010531    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:58.036176 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.036146    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\" (UniqueName: \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\") pod \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") "
Nov 03 21:31:58.036330 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.036181    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\") pod \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") "
Nov 03 21:31:58.036330 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.036266    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\") pod \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\" (UID: \"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55\") "
Nov 03 21:31:58.036725 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.036695    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca" pod "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" (UID: "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:31:58.038315 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.038289    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs" pod "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" (UID: "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55"). InnerVolumeSpecName "kube-api-access-md5xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:31:58.038677 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.038653    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca" pod "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" (UID: "d3eb0004-2902-4a9a-8bc3-2b54fc65ad55"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:31:58.138061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.137996    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md5xs\" (UniqueName: \"kubernetes.io/projected/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-kube-api-access-md5xs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:58.138061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.138023    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-xks-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:58.138061 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.138033    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55-maas-api-openshift-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:58.499138 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.499101    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:31:58.858609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.858513    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"
Nov 03 21:31:58.858609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.858570    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj" event={"ID":"d3eb0004-2902-4a9a-8bc3-2b54fc65ad55","Type":"ContainerDied","Data":"ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb"}
Nov 03 21:31:58.858802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:31:58.858617    5915 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca8b106a2e8b44b58d721eb28d379da13216339b0b5347dd81dc4150a7a091eb"
Nov 03 21:32:03.881335 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:03.881301    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:07.003369 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:32:07.003333    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:32:07.762598 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.762537    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:07.763098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.763080    5915 cpu_manager.go:414] "containerMap: removing container" podUID="d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" containerName="revoke-keys"
Nov 03 21:32:07.763098 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.763099    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" containerName="revoke-keys"
Nov 03 21:32:07.763215 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.763188    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" containerName="revoke-keys"
Nov 03 21:32:07.766680 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.766651    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.831966 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.831938    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.832132 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.831990    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-972cj\" (UniqueName: \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.832132 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.832106    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.832210 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.832133    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.868094 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.868063    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 21:32:07.868271 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.868148    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 21:32:07.868348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.868290    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 21:32:07.868348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.868294    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lnz6c\""
Nov 03 21:32:07.868348 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.868320    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 21:32:07.876375 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.876346    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:07.940924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.940888    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-972cj\" (UniqueName: \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.941224 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.941201    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.941353 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.941337    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.941596 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.941539    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.942501 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.942479    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.946760 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.946603    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:07.954413 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:07.954123    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:08.002993 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:08.002963    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-972cj\" (UniqueName: \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\") pod \"maas-api-8669f899bf-78msp\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:08.080590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:08.080478    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:08.458809 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:08.458786    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:08.460607 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:32:08.460572    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c9eb6a2_10e0_4ef4_9374_08056669354a.slice/crio-6d6c38f30c62cf3da5abcbc5a293ca538169e21d756865e8cc549fc4424d0a2f WatchSource:0}: Error finding container 6d6c38f30c62cf3da5abcbc5a293ca538169e21d756865e8cc549fc4424d0a2f: Status 404 returned error can't find the container with id 6d6c38f30c62cf3da5abcbc5a293ca538169e21d756865e8cc549fc4424d0a2f
Nov 03 21:32:08.461965 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:08.461942    5915 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:32:08.949418 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:08.949391    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" event={"ID":"1c9eb6a2-10e0-4ef4-9374-08056669354a","Type":"ContainerStarted","Data":"6d6c38f30c62cf3da5abcbc5a293ca538169e21d756865e8cc549fc4424d0a2f"}
Nov 03 21:32:09.957824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:09.957792    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" event={"ID":"1c9eb6a2-10e0-4ef4-9374-08056669354a","Type":"ContainerStarted","Data":"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"}
Nov 03 21:32:09.958214 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:09.957937    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:10.076251 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:10.076202    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" podStartSLOduration=2.22767159 podStartE2EDuration="3.076189785s" totalImagesPullingTime="848.518195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:10.07475849 +0000 UTC m=+3144.526127667" watchObservedRunningTime="2026-11-03 21:32:10.076189785 +0000 UTC m=+3144.527558961"
Nov 03 21:32:14.248808 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:14.248773    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:15.969859 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:15.969822    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:15.972870 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:15.972840    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:15.996617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:15.996588    5915 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" podUID="1c9eb6a2-10e0-4ef4-9374-08056669354a" containerName="maas-api" containerID="cri-o://2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c" gracePeriod=30
Nov 03 21:32:16.021824 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.021798    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"]
Nov 03 21:32:16.025445 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.025430    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.058566 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.058535    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"]
Nov 03 21:32:16.071129 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.071110    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-jpxkh"]
Nov 03 21:32:16.107595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.107550    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"]
Nov 03 21:32:16.120324 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.120296    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v242j\" (UniqueName: \"kubernetes.io/projected/192c38a3-78ca-46fe-b9fe-b9645cc37232-kube-api-access-v242j\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.120479 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.120334    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-metrics-certs\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.120479 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.120356    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-maas-api-tls\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.120641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.120497    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192c38a3-78ca-46fe-b9fe-b9645cc37232-openshift-service-ca\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.222015 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.221932    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192c38a3-78ca-46fe-b9fe-b9645cc37232-openshift-service-ca\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.222194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.222044    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v242j\" (UniqueName: \"kubernetes.io/projected/192c38a3-78ca-46fe-b9fe-b9645cc37232-kube-api-access-v242j\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.222194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.222079    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-metrics-certs\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.222194 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.222108    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-maas-api-tls\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.222684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.222658    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/192c38a3-78ca-46fe-b9fe-b9645cc37232-openshift-service-ca\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.224803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.224781    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-maas-api-tls\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.225019 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.224999    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/192c38a3-78ca-46fe-b9fe-b9645cc37232-metrics-certs\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.250759 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.250728    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:16.283074 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.283047    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v242j\" (UniqueName: \"kubernetes.io/projected/192c38a3-78ca-46fe-b9fe-b9645cc37232-kube-api-access-v242j\") pod \"maas-api-76d76db597-6d9c5\" (UID: \"192c38a3-78ca-46fe-b9fe-b9645cc37232\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.322685 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.322654    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\" (UniqueName: \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\") pod \"1c9eb6a2-10e0-4ef4-9374-08056669354a\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") "
Nov 03 21:32:16.322833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.322706    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\") pod \"1c9eb6a2-10e0-4ef4-9374-08056669354a\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") "
Nov 03 21:32:16.322833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.322764    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\") pod \"1c9eb6a2-10e0-4ef4-9374-08056669354a\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") "
Nov 03 21:32:16.322833 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.322805    5915 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\") pod \"1c9eb6a2-10e0-4ef4-9374-08056669354a\" (UID: \"1c9eb6a2-10e0-4ef4-9374-08056669354a\") "
Nov 03 21:32:16.323167 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.323137    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca" pod "1c9eb6a2-10e0-4ef4-9374-08056669354a" (UID: "1c9eb6a2-10e0-4ef4-9374-08056669354a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:32:16.325072 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.325039    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls" pod "1c9eb6a2-10e0-4ef4-9374-08056669354a" (UID: "1c9eb6a2-10e0-4ef4-9374-08056669354a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:32:16.325339 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.325311    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs" pod "1c9eb6a2-10e0-4ef4-9374-08056669354a" (UID: "1c9eb6a2-10e0-4ef4-9374-08056669354a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:32:16.325448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.325353    5915 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj" pod "1c9eb6a2-10e0-4ef4-9374-08056669354a" (UID: "1c9eb6a2-10e0-4ef4-9374-08056669354a"). InnerVolumeSpecName "kube-api-access-972cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:32:16.389143 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.389113    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:16.424483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.424455    5915 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-maas-api-tls\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:16.424483 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.424483    5915 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c9eb6a2-10e0-4ef4-9374-08056669354a-openshift-service-ca\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:16.424684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.424493    5915 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c9eb6a2-10e0-4ef4-9374-08056669354a-metrics-certs\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:16.424684 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.424501    5915 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-972cj\" (UniqueName: \"kubernetes.io/projected/1c9eb6a2-10e0-4ef4-9374-08056669354a-kube-api-access-972cj\") on node \"ip-10-0-141-248.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:16.765801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:16.765776    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"]
Nov 03 21:32:16.766988 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:32:16.766957    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192c38a3_78ca_46fe_b9fe_b9645cc37232.slice/crio-5c9924291c20441f5979fb576f0b5c38cfe8daa3d55c61a3a9f5537919409143 WatchSource:0}: Error finding container 5c9924291c20441f5979fb576f0b5c38cfe8daa3d55c61a3a9f5537919409143: Status 404 returned error can't find the container with id 5c9924291c20441f5979fb576f0b5c38cfe8daa3d55c61a3a9f5537919409143
Nov 03 21:32:17.003802 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.003761    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5" event={"ID":"192c38a3-78ca-46fe-b9fe-b9645cc37232","Type":"ContainerStarted","Data":"5c9924291c20441f5979fb576f0b5c38cfe8daa3d55c61a3a9f5537919409143"}
Nov 03 21:32:17.005386 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.005357    5915 generic.go:356] "Generic (PLEG): container finished" podID="1c9eb6a2-10e0-4ef4-9374-08056669354a" containerID="2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c" exitCode=0
Nov 03 21:32:17.005536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.005401    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" event={"ID":"1c9eb6a2-10e0-4ef4-9374-08056669354a","Type":"ContainerDied","Data":"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"}
Nov 03 21:32:17.005536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.005421    5915 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp"
Nov 03 21:32:17.005536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.005438    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-78msp" event={"ID":"1c9eb6a2-10e0-4ef4-9374-08056669354a","Type":"ContainerDied","Data":"6d6c38f30c62cf3da5abcbc5a293ca538169e21d756865e8cc549fc4424d0a2f"}
Nov 03 21:32:17.005536 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.005465    5915 scope.go:122] "RemoveContainer" containerID="2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"
Nov 03 21:32:17.021412 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.021395    5915 scope.go:122] "RemoveContainer" containerID="2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"
Nov 03 21:32:17.021727 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:32:17.021709    5915 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c\": container with ID starting with 2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c not found: ID does not exist" containerID="2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"
Nov 03 21:32:17.021793 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.021737    5915 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c"} err="failed to get container status \"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c\": rpc error: code = NotFound desc = could not find container \"2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c\": container with ID starting with 2f1a501f9e1212f99f5f30435bea826e50bd2b4c43781b14a785654ee6c8867c not found: ID does not exist"
Nov 03 21:32:17.168890 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.168866    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:17.171037 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.171016    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-78msp"]
Nov 03 21:32:17.682601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.682550    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c9eb6a2-10e0-4ef4-9374-08056669354a" path="/var/lib/kubelet/pods/1c9eb6a2-10e0-4ef4-9374-08056669354a/volumes"
Nov 03 21:32:17.683140 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:17.683125    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e94819cb-42b7-4770-bb1d-9402c23ea5ce" path="/var/lib/kubelet/pods/e94819cb-42b7-4770-bb1d-9402c23ea5ce/volumes"
Nov 03 21:32:18.014478 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:18.014439    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5" event={"ID":"192c38a3-78ca-46fe-b9fe-b9645cc37232","Type":"ContainerStarted","Data":"93c32c85681aa434f376af457c81d80ed35c32366b0ca740110d32030b3a2146"}
Nov 03 21:32:18.014921 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:18.014651    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:18.132373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:18.132326    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5" podStartSLOduration=2.243020778 podStartE2EDuration="3.132311337s" totalImagesPullingTime="889.290559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:18.132061274 +0000 UTC m=+3152.583430450" watchObservedRunningTime="2026-11-03 21:32:18.132311337 +0000 UTC m=+3152.583680516"
Nov 03 21:32:24.026714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:24.026687    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76d76db597-6d9c5"
Nov 03 21:32:24.221839 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:24.221806    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:24.800390 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:24.800357    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:25.243922 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:25.243891    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:28.201878 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:28.201843    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:28.366980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:28.366940    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:28.519373 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:28.519289    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:29.870971 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:29.870938    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:31.472573 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:31.472537    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:31.653629 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:31.653598    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:33.461428 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:33.461395    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:33.619466 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:33.619429    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:33.765619 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:33.765504    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:37.003597 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:32:37.003539    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:32:38.412666 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:38.412632    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:43.320411 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:43.320350    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:48.110054 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:48.110025    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:51.350764 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:51.350730    5915 scope.go:122] "RemoveContainer" containerID="071129e6d39656375aeee09087362e930dbc9a0d5fd5d2977f1e1704c6d832c2"
Nov 03 21:32:53.212336 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:53.212304    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:32:58.370682 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:32:58.370647    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:03.217055 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:03.217014    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:07.003099 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:33:07.003064    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:33:07.452003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:07.451973    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:07.633595 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:07.625728    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:08.032768 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:08.032738    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:12.256012 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:12.255979    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:13.782400 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:13.782368    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:14.058617 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:14.058534    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:14.201630 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:14.201586    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:14.592226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:14.592188    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:15.323811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:15.323777    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:16.538283 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:16.538250    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:17.143321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:17.143293    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:17.334257 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:17.334222    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:21.195724 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:21.195690    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:21.345720 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:21.345685    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:23.770979 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:23.770948    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:23.927973 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:23.927944    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:24.074448 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:24.074316    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:24.332618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:24.332535    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:24.517172 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:24.517102    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:28.727106 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:28.727073    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:30.221378 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:30.221338    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:33.525083 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:33.525040    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:37.003362 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:33:37.003324    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:33:38.322321 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:38.322258    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:40.836899 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:40.836868    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:43.510869 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:43.510830    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:46.046986 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:46.046953    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"]
Nov 03 21:33:46.096311 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:46.096281    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-78a4f552-sl7bs"]
Nov 03 21:33:47.680599 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:47.680538    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a0769a3-5d8d-4591-907f-d093396ebbe0" path="/var/lib/kubelet/pods/9a0769a3-5d8d-4591-907f-d093396ebbe0/volumes"
Nov 03 21:33:48.230065 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:48.230012    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:51.379372 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:51.379337    5915 scope.go:122] "RemoveContainer" containerID="d187fcb8637ba6c5cf53403fc644cd29ea9c9570973e2aa4d048f81ddd3a1447"
Nov 03 21:33:53.012819 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:53.012780    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:33:57.726293 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:33:57.726183    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:02.522832 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:02.522794    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:07.003657 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:34:07.003618    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:34:07.228316 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:07.228284    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:12.018169 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:12.018103    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:14.547001 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:14.546943    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:16.745749 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:16.745711    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:21.511668 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:21.511620    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:25.151119 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:25.151078    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:25.293163 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:25.293111    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:25.444111 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:25.443998    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:26.302610 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:26.302537    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:31.111199 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:31.111143    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:35.891601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:35.891522    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:37.003992 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:34:37.003955    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:34:40.730803 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:40.730771    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:45.520057 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:45.520028    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:46.444605 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:46.444540    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:34:46.452367 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:46.452340    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:34:46.458919 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:46.458900    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:34:46.466090 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:46.466074    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:34:50.318604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:50.318519    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:51.406695 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:51.406665    5915 scope.go:122] "RemoveContainer" containerID="038896ae5df50dafe3f7dc3dcc5cb0530c7a525bf96b51ed06c7c8ac91c16dce"
Nov 03 21:34:55.118476 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:55.118439    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:34:59.915234 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:34:59.915174    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:04.048626 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:04.048584    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:04.856511 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:04.856480    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:07.003440 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:35:07.003407    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:35:10.018740 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:10.018708    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:14.906593 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:14.906543    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.063383 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:15.063352    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.813537 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:15.813505    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:15.962226 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:15.962191    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:16.142241 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:16.142154    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:16.733455 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:16.733424    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:19.942270 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:19.942239    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:24.136394 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:24.136360    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:25.650255 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:25.650159    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:25.918058 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:25.917976    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:26.703887 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:26.703855    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:32.384038 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:32.384007    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:32.528752 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:32.528721    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:32.702269 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:32.702230    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:35.576434 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:35.576400    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:35:37.002956 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:35:37.002919    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:35:48.048601 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:48.048537    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"]
Nov 03 21:35:48.095292 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:48.095264    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-bp648"]
Nov 03 21:35:49.680359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:49.680318    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35c50605-8214-49e5-b7d6-b7a3785fab38" path="/var/lib/kubelet/pods/35c50605-8214-49e5-b7d6-b7a3785fab38/volumes"
Nov 03 21:35:51.430291 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:35:51.430255    5915 scope.go:122] "RemoveContainer" containerID="318c4fc7218dc6bacfee69abca500d1b14df792d148bf0746b16daedad39b928"
Nov 03 21:36:07.001932 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:36:07.001899    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:36:15.254377 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:15.254339    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:20.395585 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:20.394201    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:21.047758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:21.047726    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"]
Nov 03 21:36:21.099736 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:21.099691    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-btv86"]
Nov 03 21:36:21.679882 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:21.679851    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8c37113-33be-4faf-b6e6-6e0ec4c01e56" path="/var/lib/kubelet/pods/a8c37113-33be-4faf-b6e6-6e0ec4c01e56/volumes"
Nov 03 21:36:35.921018 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:35.920982    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:37.003204 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:36:37.003167    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:36:42.972618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:42.972586    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cdd7cfd6-c9vgv_dddc4384-4ea1-43ae-917d-dec91c54d2e3/authorino/0.log"
Nov 03 21:36:44.034974 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:44.034942    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:44.218590 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:44.218535    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:36:51.463419 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:51.463385    5915 scope.go:122] "RemoveContainer" containerID="cc74f628f3b1399d1465e5310931642de26c87da44331f03d1090410953c9399"
Nov 03 21:36:58.049112 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:58.049040    5915 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"]
Nov 03 21:36:58.098641 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:58.098612    5915 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d4735150-zv2kj"]
Nov 03 21:36:59.680656 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:36:59.680626    5915 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3eb0004-2902-4a9a-8bc3-2b54fc65ad55" path="/var/lib/kubelet/pods/d3eb0004-2902-4a9a-8bc3-2b54fc65ad55/volumes"
Nov 03 21:37:05.267117 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:05.267086    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:37:06.223856 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:06.223825    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zrrbs_623d2762-055f-48d9-b81b-28664f039e64/proxy/0.log"
Nov 03 21:37:06.986075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:06.986039    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d76db597-6d9c5_192c38a3-78ca-46fe-b9fe-b9645cc37232/maas-api/0.log"
Nov 03 21:37:07.003259 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:37:07.003225    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:37:09.680840 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:09.680815    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cdd7cfd6-c9vgv_dddc4384-4ea1-43ae-917d-dec91c54d2e3/authorino/0.log"
Nov 03 21:37:10.472495 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:10.472456    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-s7w5j_bd891d58-0ffd-4e01-a8d9-2710445a5880/manager/0.log"
Nov 03 21:37:11.540071 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:11.540038    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-zxvm4_4aa1413a-79c3-48a2-816b-71790bbb6727/manager/0.log"
Nov 03 21:37:12.575429 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:12.575403    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-8lj8l_cb47871f-c00a-4e19-897d-3c89fa082b2e/discovery/0.log"
Nov 03 21:37:12.813687 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:12.813657    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68d884f4c6-gwf8q_d7ac0f39-0dc9-403d-b4bb-a324a82c4958/kube-auth-proxy/0.log"
Nov 03 21:37:13.489995 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:13.489967    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s_fb0d68f7-486e-44d7-a324-11f4ea590bb4/istio-proxy/0.log"
Nov 03 21:37:15.053525 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:15.053498    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8_1db44974-2a96-495d-bccd-290898b0c2af/istio-proxy/0.log"
Nov 03 21:37:15.282265 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:15.282236    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z_d48e0d98-751a-41ff-b3ca-a80292e6c9f4/istio-proxy/0.log"
Nov 03 21:37:15.511529 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:15.511497    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-xxsms_4d76e840-1ed9-42ef-b9d1-2c889b992835/istio-proxy/0.log"
Nov 03 21:37:16.407370 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:16.407342    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5769c487dc-z9pqs_4c2480e6-eaa3-402a-b3fc-2c63f26ff172/router/0.log"
Nov 03 21:37:17.176857 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:17.176824    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s_999ea8f5-7307-4421-8caf-06d86bacc5a7/main/0.log"
Nov 03 21:37:18.319155 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:18.319131    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-p6hxw_4e78f464-2e1e-460b-8ea5-ade9414e73ea/main/0.log"
Nov 03 21:37:18.556458 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:18.556427    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m_e2e2f602-9c96-44f1-9bce-b60c8c4abab4/main/0.log"
Nov 03 21:37:19.685801 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:19.685772    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cdd7cfd6-c9vgv_dddc4384-4ea1-43ae-917d-dec91c54d2e3/authorino/0.log"
Nov 03 21:37:35.604855 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:35.604820    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:36.267714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:36.267682    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:37.002783 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:37:37.002743    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:37:44.318128 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:44.318090    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:44.463618 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:44.463589    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:50.060177 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:50.060144    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:37:51.486637 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:51.486608    5915 scope.go:122] "RemoveContainer" containerID="536d6c7f62948299059d7cb24d971f138bfab7d4204bbde8929c7dbfad6b8e34"
Nov 03 21:37:56.667120 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:56.667087    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:37:56.810302 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:37:56.810267    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:01.140822 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:01.140795    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zrrbs_623d2762-055f-48d9-b81b-28664f039e64/proxy/0.log"
Nov 03 21:38:01.809171 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:01.809133    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d76db597-6d9c5_192c38a3-78ca-46fe-b9fe-b9645cc37232/maas-api/0.log"
Nov 03 21:38:04.245609 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:04.245575    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cdd7cfd6-c9vgv_dddc4384-4ea1-43ae-917d-dec91c54d2e3/authorino/0.log"
Nov 03 21:38:05.026516 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:05.026486    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-s7w5j_bd891d58-0ffd-4e01-a8d9-2710445a5880/manager/0.log"
Nov 03 21:38:05.985235 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:05.985198    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-zxvm4_4aa1413a-79c3-48a2-816b-71790bbb6727/manager/0.log"
Nov 03 21:38:06.799510 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:06.799484    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-8lj8l_cb47871f-c00a-4e19-897d-3c89fa082b2e/discovery/0.log"
Nov 03 21:38:07.002766 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:38:07.002731    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:38:07.035488 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:07.035459    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68d884f4c6-gwf8q_d7ac0f39-0dc9-403d-b4bb-a324a82c4958/kube-auth-proxy/0.log"
Nov 03 21:38:07.692075 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:07.692037    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-6bk9s_fb0d68f7-486e-44d7-a324-11f4ea590bb4/istio-proxy/0.log"
Nov 03 21:38:09.151935 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:09.151908    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-nwhf8_1db44974-2a96-495d-bccd-290898b0c2af/istio-proxy/0.log"
Nov 03 21:38:09.373863 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:09.373831    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-wdg5z_d48e0d98-751a-41ff-b3ca-a80292e6c9f4/istio-proxy/0.log"
Nov 03 21:38:09.590688 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:09.590655    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-xxsms_4d76e840-1ed9-42ef-b9d1-2c889b992835/istio-proxy/0.log"
Nov 03 21:38:10.477545 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:10.477514    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5769c487dc-z9pqs_4c2480e6-eaa3-402a-b3fc-2c63f26ff172/router/0.log"
Nov 03 21:38:11.035841 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:11.035811    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-mm96s_999ea8f5-7307-4421-8caf-06d86bacc5a7/main/0.log"
Nov 03 21:38:12.186719 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:12.186661    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-p6hxw_4e78f464-2e1e-460b-8ea5-ade9414e73ea/main/0.log"
Nov 03 21:38:12.405926 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:12.405890    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-2k56m_e2e2f602-9c96-44f1-9bce-b60c8c4abab4/main/0.log"
Nov 03 21:38:25.742003 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:25.741869    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:35.966281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:35.966242    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:36.607943 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:36.607909    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:38:37.003116 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:38:37.003084    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:38:51.515359 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:38:51.515329    5915 scope.go:122] "RemoveContainer" containerID="1cae02fb1e0954e02874d71d20b023fb8a9e728de257bb19edf89a7a86385b82"
Nov 03 21:39:07.003576 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:39:07.003522    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:39:12.875356 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:12.875323    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:39:13.435789 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:13.435757    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:39:26.095604 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:26.095544    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:36.296173 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:36.296139    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:39:37.003954 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:39:37.003907    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:39:46.487113 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:46.487081    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:39:46.501004 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:46.500979    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-marketplace_6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a4hkjw_e2233017-06ef-4c6e-bece-94c4128c467d/extract/0.log"
Nov 03 21:39:46.503680 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:46.503662    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:39:46.514400 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:39:46.514383    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-xv2zb_6d37aa0c-7893-407a-a52a-46d8a1ffd00e/console-operator/2.log"
Nov 03 21:40:07.002936 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:40:07.002896    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:40:23.434524 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:23.434493    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:40:24.006220 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:24.006189    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6778fc7d44-hlt4w_30165c7a-d42c-464b-81b0-800805bb5f3d/manager/0.log"
Nov 03 21:40:25.302864 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:25.302825    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:25.457643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:25.457609    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:26.428202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:26.428162    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:35.344980 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:35.344946    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xjbt9_5713f5a0-c8bf-44c1-aeff-c9c580746c45/global-pull-secret-syncer/0.log"
Nov 03 21:40:35.871184 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:35.871147    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:35.967643 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:35.967617    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-dnb6d_30b966c5-d3d0-4743-a041-fe5f0f196fd1/connection-checker/0.log"
Nov 03 21:40:36.386281 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:36.386254    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-l9wgt_90155d8a-eff2-4b22-ac1b-975bffe1d644/connection-checker/0.log"
Nov 03 21:40:36.648741 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:36.648651    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:36.805549 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:36.805523    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-w4kw4_2e5e0dcf-8842-4b4a-bc4b-d11e653865fb/connection-checker/0.log"
Nov 03 21:40:37.003302 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:40:37.003268    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"
Nov 03 21:40:37.282477 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:37.282389    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log"
Nov 03 21:40:37.321694 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:37.321664    5915 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cks2d_ff27c4bf-c1d8-47f8-9737-24510e201615/konnectivity-agent/0.log" limit=1024
Nov 03 21:40:39.089908 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:39.089879    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-248.us-west-1.compute.internal_fd3e66eb7ec8c0db50f85619df6599ce/haproxy/0.log"
Nov 03 21:40:43.457652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:43.457621    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-78cdd7cfd6-c9vgv_dddc4384-4ea1-43ae-917d-dec91c54d2e3/authorino/0.log"
Nov 03 21:40:44.151202 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.151170    5915 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"]
Nov 03 21:40:44.151652 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.151635    5915 cpu_manager.go:414] "containerMap: removing container" podUID="1c9eb6a2-10e0-4ef4-9374-08056669354a" containerName="maas-api"
Nov 03 21:40:44.151714 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.151655    5915 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c9eb6a2-10e0-4ef4-9374-08056669354a" containerName="maas-api"
Nov 03 21:40:44.151758 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.151750    5915 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c9eb6a2-10e0-4ef4-9374-08056669354a" containerName="maas-api"
Nov 03 21:40:44.154924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.154908    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.185481 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.185455    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9khz\" (UniqueName: \"kubernetes.io/projected/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-kube-api-access-v9khz\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.185638 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.185487    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-sys\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.185715 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.185666    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-podres\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.185780 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.185745    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-proc\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.185828 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.185779    5915 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-lib-modules\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.202729 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.202676    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-s7w5j_bd891d58-0ffd-4e01-a8d9-2710445a5880/manager/0.log"
Nov 03 21:40:44.206837 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.206817    5915 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fd9m6\"/\"default-dockercfg-9chns\""
Nov 03 21:40:44.206924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.206862    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd9m6\"/\"openshift-service-ca.crt\""
Nov 03 21:40:44.206924 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.206902    5915 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd9m6\"/\"kube-root-ca.crt\""
Nov 03 21:40:44.260216 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.260190    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"]
Nov 03 21:40:44.286409 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286388    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-proc\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286416    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-lib-modules\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286439    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9khz\" (UniqueName: \"kubernetes.io/projected/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-kube-api-access-v9khz\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286455    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-sys\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286513 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286500    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-proc\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286707 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286529    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-lib-modules\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286707 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286593    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-sys\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286707 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286640    5915 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-podres\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.286811 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.286725    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-podres\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.347534 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.347509    5915 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9khz\" (UniqueName: \"kubernetes.io/projected/b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a-kube-api-access-v9khz\") pod \"perf-node-gather-daemonset-6s94t\" (UID: \"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.467228 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.467201    5915 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:44.846941 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:44.846916    5915 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"]
Nov 03 21:40:44.849073 ip-10-0-141-248 kubenswrapper[5915]: W1103 21:40:44.849045    5915 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb6b82f73_d0d7_4fe8_8d91_37e9c6e6579a.slice/crio-ae63dc1c63538fd43c1acbb0ade26bfcb7a187cb088b3942a9b3efa2449a557b WatchSource:0}: Error finding container ae63dc1c63538fd43c1acbb0ade26bfcb7a187cb088b3942a9b3efa2449a557b: Status 404 returned error can't find the container with id ae63dc1c63538fd43c1acbb0ade26bfcb7a187cb088b3942a9b3efa2449a557b
Nov 03 21:40:45.141676 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:45.141551    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t" event={"ID":"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a","Type":"ContainerStarted","Data":"ce8909205969c55d119aadd4f658f614e4aef473239f32f662f7c0250a394e8b"}
Nov 03 21:40:45.141834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:45.141681    5915 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t" event={"ID":"b6b82f73-d0d7-4fe8-8d91-37e9c6e6579a","Type":"ContainerStarted","Data":"ae63dc1c63538fd43c1acbb0ade26bfcb7a187cb088b3942a9b3efa2449a557b"}
Nov 03 21:40:45.141834 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:45.141712    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:45.258181 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:45.258128    5915 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t" podStartSLOduration=1.258106952 podStartE2EDuration="1.258106952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:45.256005287 +0000 UTC m=+3659.707374465" watchObservedRunningTime="2026-11-03 21:40:45.258106952 +0000 UTC m=+3659.709476223"
Nov 03 21:40:45.921616 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:45.921584    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-zxvm4_4aa1413a-79c3-48a2-816b-71790bbb6727/manager/0.log"
Nov 03 21:40:51.158917 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:51.158886    5915 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-6s94t"
Nov 03 21:40:51.451213 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:51.451183    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-lbb9n_f76fe3d1-1a83-4e2b-867c-792eba90309f/cluster-monitoring-operator/0.log"
Nov 03 21:40:56.562974 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:56.562943    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rqrcb_2baeac45-94a5-4d6b-85ea-a257c2f6482f/node-exporter/0.log"
Nov 03 21:40:56.977135 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:56.977110    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rqrcb_2baeac45-94a5-4d6b-85ea-a257c2f6482f/kube-rbac-proxy/0.log"
Nov 03 21:40:57.392453 ip-10-0-141-248 kubenswrapper[5915]: I1103 21:40:57.392378    5915 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rqrcb_2baeac45-94a5-4d6b-85ea-a257c2f6482f/init-textfile/0.log"
Nov 03 21:41:07.003902 ip-10-0-141-248 kubenswrapper[5915]: E1103 21:41:07.003863    5915 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tph4g" containerName="extractor"